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1. 


Introduction 


Two  artificially  intelligent  (Al)  computer  agents  begin  to  play  a  game  of  chess,  and 
the  following  conversation  ensues: 

5i:  Do  you  know  the  rules  of  chess? 

S2:  Yes. 

Si:  Then  you  know  whether  White  has  a  forced  initial  win  or  not. 

52!  Upon  reflection,  1  realize  that  I  must. 

Si:  Then  there  is  no  reason  to  pJay. 

S2:  No. 

Both  agents  are  state-of-the-art  constructions,  incorporating  the  latest  Al  research 
in  chess  playing,  natural- language  understanding,  planning,  etc.  But  because  of  the  over¬ 
whelming  combinatorics  of  chess,  neither  they  nor  the  fastest  foreseeable  computers  would 
be  able  to  search  the  entire  game  tree  to  find  out  whether  White  has  a  forced  win.  Why 
then  do  they  come  to  such  an  odd  conclusion  about  their  own  knowledge  of  the  game? 

The  chess  scenario  is  an  anecdotal  example  of  the  way  inaccurate  cognitive  models 
can  lead  to  behavior  that  is  less  than  intelligent  in  artificial  agents.  In  this  case,  the  agents’ 
model  of  belief  is  not  correct.  They  make  the  assumption  that  an  agent  actually  knows 
all  the  consequences  of  his  beliefs.  Si  knows  that  chess  is  a  finite  game,  and  thus  reasons 
that,  in  principle,  knowing  the  rules  of  chess  is  all  that  is  required  to  figure  out  whether 
White  has  a  forced  initial  win.  After  learning  that  ^2  does  indeed  know  the  rules  of  chess, 
he  comes  to  the  erroneous  conclusion  that  S2  also  knows  this  particular  consequence  of 
the  rules.  And  52  himself,  reflecting  on  his  own  knowledge  in  the  same  manner,  arrives  at 
the  same  conclusion,  even  though  in  actual  fact  he  could  never  carry  out  the  computations 
necessary  to  demonstrate  it. 
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We  call  the  assumption  that  an  agent  knows  all  logical  consequences  of  his  beliefs 
consequent/a/  closure.  This  assumption  is  clearly  not  warranted  for  either  mechanical  or 
human  agents,  because  some  consequences,  although  they  are  logically  correct,  may  not 
be  computationally  feasible  to  derive.  This  is  in  fact  illustrated  by  the  chess  scenario. 
Unfortunately,  the  best  current  formal  models  of  belief  on  which  AI  systems  are  based 
have  a  possible-wo^ds  semantics,  and  one  of  the  inherent  properties  of  these  models  is 
consequential  closure.  While  such  models  are  good  at  predicting  what  consequences  an 
agent  could  possibly  derive  from  his  beliefs,  they  are  not  capable  of  predicting  what  an 
agent  actually  believes,  given  that  the  agent  may  have  resource  limitations  impeding  the 
derivation  of  the  consequences  of  his  beliefs. 

The  chess  scenario  illustrates  one  source  of  logical  incompleteness  in  belief  deriva¬ 
tion,  namely,  an  agent  may  not  have  enough  computational  resources  to  actually  derive 
some  result.  We  will  identify  several  others  in  Section  2,  by  presenting  a  problem  in  belief 
representation  that  we  have  called  the  Not-So- Wise-Man  Problem,  a  variation  of  the  famil¬ 
iar  Wise  Man  Puzzle.  Not  surprisingly,  this  problem  involves  reasoning  about  beliefs  an 
agent  does  not  have,  even  though  they  are  logical  consequences  of  his  beliefs.  The  repre¬ 
sentational  problems  posed  by  the  chess  scenario  and  the  not-so-wise-man  problem  cannot 
be  solved  within  the  framework  of  any  model  of  belief  that  assumes  consequential  closure. 


In  this  paper  we  introduce  a  new  formal  model  of  belief,  called  the  deduction  mode/, 
for  representing  situations  in  which  belief  derivation  is  logically  incomplete.  Its  main  fea¬ 
ture  is  that  it  is  a  symbol-processing  model:  beliefs  are  taken  to  be  expressions  in  some 
internal  or  “mental”  language,  and  an  agent  reasons  about  his  beliefs  by  manipulating  these 
syntactic  objects.  Because  the  derivation  of  consequences  of  beliefs  is  represented  explicitly 
as  a  syntactic  process  in  this  model,  it  is  possible  to  take  into  account  the  fact  that  agents 
can  derive  some  of  the  logically  possible  consequences,  but  in  many  cases  not  all  of  them. 
When  the  process  of  belief  derivation  is  logically  incomplete,  the  deduction  model  does  not 
have  the  property  of  consequential  closure. 

Symbol-processing  models  of  belief  in  themselves  are  not  new  (see,  for  e.\ainple, 
Fodor  [10],  Lycan  [23],  and  Moore  and  Hendrix  [31]  for  some  philosophical  underpinnings, 
and  McCarthy  [26],  Perils  [33],  and  Konolige  [19]  for  Al  approaches).  The  deduction  model 
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presented  here  differs  significantly  from  previous  approaches,  however,  in  two  respects. 
First,  it  is  a  formal  model:  beliefs  are  represented  in  a  mathematical  framework  called  a 
deduction  structure.  The  properties  of  the  deduction  model  can  be  examined  with  some 
preciseness,  and  we  do  so  in  Section  3.  Second,  we  have  found  sound  and  complete  logics  for 
the  deduction  model.  One  of  these,  B,  is  presented  in  Section  4,  and  used  in  the  solution 
of  the  problems  in  Section  5.  An  important  property  of  the  deductive  belief  logic  B  is 
that  it  can  serve  as  a  basis  for  building  computer  agents  that  reason  about  belief.  We 
have  been  able  to  find  a  number  of  interesting  proof  methods  for  B  that  have  reasonable 
computational  properties.  Although  the  exposition  of  these  methods  is  beyond  the  scope  of 
this  paper,  at  the  appropriate  points  we  will  show  how  the  design  of  the  logic  was  influenced 
by  computational  considerations. 

The  nature  of  the  deduction  model  and  its  logic  B  is  further  analyzed  by  comparing 
B  to  modal  logics  based  on  a  possible*worlds  semantics  in  Section  6.  An  important  result 
is  that  the  deduction  model  exhibits  a  correspondence  property:  in  the  limit  of  logically 
complete  deduction,  8  reduces  to  a  modal  logic  with  possible* worlds  semantics.  Thus  the 
deduction  model  dominates  the  possible-worlds  model,  while  retracting  the  assumption  of 
consequential  closure. 


The  material  for  this  paper  was  abstracted  from  the  author’s  dissertation  work 
(Konolige  [21]).  Because  of  the  limited  scope  of  this  paper,  we  are  not  able  to  do  more 
than  mention  in  passing  several  interesting  topics  that  are  a  part  of  the  deduction  model  and 
its  logics.  Among  these  are  efficient  proof  methods,  the  formal  semantics  and  completeness 
proofs,  extensions  to  B  that  permit  quantifying-in,  and  introspection  properties  (beliefs 
about  one’s  own  beliefs).  Interested  readers  can  consult  the  dissertation  itself  for  a  fuller 
exposition. 
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2. 


Two  Problems  in  the  Representation  of  Belief 


In  this  section  we  introduce  three  ways  in  which  an  a^ent  may  be  incomplete  in 
reasoning  from  his  beliefs;  resource-y/mited  mcompleteness,  fundameDtal  logical  incom- 
pleteness,  and  relevance  mcompleteness.  We  argue  that  it  is  important  for  AI  systems  that 
reason  about  belief  to  be  able  to  represent  each  of  these,  and  offer  two  anecdotal  problems 
to  support  this  contention. 


The  Chess  Problem.  Suppose  an  agent  knows  the  rules  of  chess.  It  does 
not  necessarily  follow  that  he  knows  whether  White  has  a  winning  strategy 
or  not. 


The  chess  problem,  on  the  face  of  it,  seems  hardly  to  be  a  representational  problem 
at  all.  Certainly  its  statement  is  true:  no  agent,  human  or  otherwise,  can  possibly  follow 
out  all  the  myriad  lines  of  chess  play  allowed  by  the  rules  to  determine  whether  White  has 
a  strategy  that  will  always  win.  What  kind  of  model  of  belief  would  lead  us  to  expect  an 
agent  to  know  whether  White  has  a  winning  strategy?  As  we  stated  in  the  introduction, 
any  model  that  does  not  take  resource  limitations  into  account  in  representing  an  agent’s 
reasoning  about  the  consequences  of  lus  beliefs  has  this  behavior.  Within  such  a  model,  we 
could  establish  the  following  line  of  argument. 

Chess  is  a  Unite  game,^  and  so  it  is  possible,  in  theory,  to  construct  a  complete,  Unite 
game  tree  for  chess,  given  the  rules  of  the  game.  The  question  of  Whitens  having 
a  winning  strategy  is  a  property  of  this  Bnite  game  tree.  If  for  every  counter 
Black  makes,  White  has  a  move  that  will  lead  to  a  win,  then  White  has  a  winning 
strategy.  Thus,  White’s  having  a  winning  strategy  is  a  consequence  of  the  rules  of 

^  The  finiteness  of  cheu  is  assured  by  the  rule  that,  if  50  moves  occur  without  a  pawn  advance  or  piece 
capture,  the  game  is  a  draw. 
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chess  that  can  be  derived  in  a  finite  number  of  simple  steps.  If  an  agent  believes 
all  the  logical  consequences  of  bis  beliefs,  then  an  agent  who  knows  the  rules  of 
chess  will,  by  the  reasoning  just  given,  also  know  whether  White  has  a  winning 
strategy  or  not. 

The  chess  problem  is  thus  a  problem  in  representing  reasoning  about  beliefs  in  the 
face  of  resource  limitations.  The  inference  steps  themselves  are  almost  trivial;  it  is  a  simple 
matter  to  show  that  a  move  is  legal,  and  hence  to  construct  any  position  that  follows  a  legal 
move  from  a  given  position.  But  while  the  individual  inferences  are  easy,  the  number  of 
them  required  to  figure  out  whether  White  has  a  forced  win  is  astronomical  and  beyond  the 
computational  abilities  of  any  agent.  We  call  this  behavior  resource-bm/ted  incompleteness. 
A  suitable  model  of  belief  must  be  able  to  represent  situations  in  which  an  agent  possesses 
the  inferential  capability  to  derive  some  consequence  of  his  beliefs,  but  simply  does  not 
have  the  computational  resources  to  do  so. 

The  NoT-SO- Wise-Man  Problem.  a  king,  wishing  to  know  which  of  his 
three  advisors  is  the  wisest,  paints  a  white  dot  on  each  of  their  foreheads, 
tells  them  there  is  at  least  one  white  dot,  and  asks  them  to  tell  him  the  color 
of  their  own  spots.  After  a  while  the  first  replies  that  he  doesn’t  know;  the 
second,  on  hearing  this,  also  says  he  doesn’t  know.  The  third  then  responds, 

“I  also  don’t  know  the  color  of  my  spot;  but  if  the  second  of  us  were  wiser,  I 
would  know  it.” 


The  not-so- wise- man  problem  is  a  variation  of  the  classic  Wise  Man  Puzzle,  which 
McCarthy  (in  [24]  and  [25])  has  used  extensively  as  a  test  of  models  of  knowledge.  In  the 
classic  version,  the  third  wise  man  figures  out  from  the  replies  of  the  other  two  that  his 
spot  must  be  white.  The  “puzzle”  part  is  to  generate  the  reasoning  employed  by  the  third 
wise  man.  The  reasoning  involved  is  really  quite  comple.x  and  hinges  on  the  ability  of  the 
wise  men  to  reason  about  one  another’s  beliefs.  To  convince  themselves  of  this,  readers 
who  have  never  tried  before  may  be  interested  in  attempting  to  solve  it  before  reading  the 
solution  below. 
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Solution  to  the  Wise  Man  Puzzle:  the  third  wise  man  reasons:  “Suppose  my  spot 
were  black.  Then  the  second  of  us  would  know  that  his  own  spot  was  white,  since 
he  would  know  that,  if  it  were  black,  the  first  of  us  would  have  seen  two  black 
spots  and  would  have  known  his  own  spot's  color.  Since  both  answered  that  they 
had  no  knowledge  of  their  own  spot's  color,  my  spot  must  be  white.” 


The  difficulty  behind  this  puzzle  seems  to  lie  in  the  nature  of  the  third  wise  man’s 
reasoning  about  the  first  two’s  beliefs.  Not  only  must  he  pose  a  hypothetical  situation 
(Suppose  my  spot  were  black),  but  he  must  then  reason  within  that  situation  about  what 
conclusions  the  second  wise  man  would  come  to  after  hearing  the  first  wise  man’s  response. 
This  in  turn  means  that  he  must  reason  about  the  second  wise  man’s  reasoning  about  the 
first  wise  man’s  beliefs,  as  revealed  by  his  reply  to  the  king.  Reasoning  about  beliefs  about 
beliefs  about  beliefs...  we  call  reasoning  about  iterated  or  nested  beliefs.  It  can  quickly 
become  confusing,  especially  when  there  are  conditions  present  concerning  what  an  agent 
does  not  believe. 

In  the  Wise  Man  Puzzle,  nested  belief  contributes  to  the  complexity  of  the  reasoning 
involved.  The  third  wise  man  must  reason  about  what  the  second  wise  man  does  not  know 
(the  color  of  his  own  spot);  in  doing  this,  he  must  also  reason  about  the  second  wise  man’s 
reasoning  about  what  the  third  wise  man  does  not  know  (the  color  of  bis  own  spot).  It 
is  particularly  annoying  and  troublesome  to  keep  track  of  who  believes  what  after  several 
occurrences  of  not-believing  in  a  statement  of  nested  belief.  Because  human  agents  find 
it  so  difficult,  the  Wise  Man  Puzzle  is  thought  to  be  a  good  test  of  the  competence  of 
any  model  of  belief.  If  one  can  state  the  solution  to  the  puzzle  within  the  framework  of 
Model  X,  so  the  reasoning  goes,  then  Model  X  Is  at  least  good  enough  to  show  what  might 
conceivably,  be  concluded  by  agents  in  complicated  situations  involving  nested  beliefs. 

It  is  possible  to  solve  the  Wise  Man  Puzzle  within  the  confines  of  belief  models  that 
assume  consequential  closure  (see,  e.g.,  McCarthy  [24],  [25]  or  Sato  [38]).  Such  models 
make  the  assumption  that  every  agent  believes  other  agents’  beliefs  are  closed  under  logical 
consequence,  and  so  on  to  arbitrary  depths  of  belief  nesting.  While  this  is  an  accurate 
assumption  if  one  is  trying  to  model  the  competence  of  ideal  agents  (which  is  what  the 
Wise  Man  Puzzle  seeks  to  verify),  it  cannot  represent  interesting  ways  in  which  reasoning 
about  complicated  nested  beliefs  might  fail  for  a  less-than-ideal  agent.  This  is  the  import 
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of  the  not-so-wise-man  problem.  FVom  the  reply  of  the  third  wise  man,  it  appears  that 
the  second  wise  man  lacks  the  ability  to  deduce  all  the  consequences  of  his  beliefs.  The 
representational  problem  posed  is  to  devise  interesting  ways  in  which  the  second  wise  man 
fails  to  be  an  ideal  agent,  and  then  show  how  the  third  wise  man  can  represent  this  failure 
and  reply  as  he  does. 

The  not-so-wise-man  problem  does  not  seem  to  fall  into  the  category  of  resource- 
limited  Incompleteness  mentioned  in  the  chess  problem,  since  the  computational  require¬ 
ments  of  the  inferences  are  not  particularly  acute.  We  can  identify  at  least  two  other  types 
of  incompleteness  (there  may  certainly  be  more)  that  are  interesting  here  and  would  be 
useful  to  represent.  In  one  of  these,  the  second  wise  man  may  have  incomplete  inferential 
procedures  for  reasoning  about  the  other  wise  men’s  beliefs,  especially  if  tricky  combina¬ 
tions  of  not-be/ieving  are  present.  Suppose,  for  instance,  the  second  wise  man  were  to  see 
a  black  spot  on  the  third  wise  man,  and  a  white  spot  on  the  hrst  wise  man  (this  is  the 
hypothetical  situation  set  up  by  the  third  wise  man  in  solving  the  classic  puzzle).  If  he 
were  an  ideal  agent,  he  would  conclude  from  the  first  wise  man’s  reply  that  his  own  spot 
must  be  white  (by  reasoning:  if  mine  were  not  white,  the  first  of  us  would  have  seen  two 
black  spots  and  so  c/aimed  his  own  as  white).  But  he  may  fail  to  do  this  because  his  rules 
for  reasoning  about  the  beliefs  of  the  first  wise  man  simply  are  not  powerful  enough.  For 
example,  he  might  never  consider  the  strategy  of  assuming  that  his  spot  was  black,  and 
then  asking  himself  what  the  first  wise  man  would  have  said.  In  this  case,  the  second  wise 
man’s  inferential  process,  even  when  given  adequate  resources,  is  just  not  powerful  enough 
in  terms  of  its  ability  to  arrive  at  simple  logical  conclusions.  To  apply  an  analogy  from 
high-school  algebra:  a  student  who  is  confronted  with  the  equation  x  +  a  =  6  and  asked 
to  solve  for  x  won’t  be  able  to  do  so  if  he  doesn’t  know  the  rule  that  subtracting  equals 
from  each  side  leaves  the  equation  valid.  It  is  not  that  the  student  lacks  sufficient  mental 
resources  of  time  or  memory  to  solve  this  problem;  rather,  his  rules  of  inference  for  dealing 
with  equational  theories  are  logically  incomplete.  To  contrast  this  type  of  incompleteness 
with  the  resource-limited  incompleteness  described  in  the  chess  problem,  we  call  it  funda¬ 
mental  logical  incompleteness. 


Another  way  in  which  the  not-so-wise-man  might  fail  to  draw  conclusions  is  if  he 
were  to  make  an  erroneous  decision  as  to  what  information  might  be  relevant  to  solving 
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his  problem.  Although  the  Wise  Man  Puzzle  has  a  fairly  abstract  setting,  it  is  reasonable 
to  suppose  that  actual  agents  confronted  with  this  problem  would  have  a  fair  number  of 
extraneous  beliefs  that  they  would  exclude  from  consideration.  For  example,  the  not-so- 
wise-man  might  be  privy  to  the  castle  rumor  mill,  and  therefore  believe  that  the  first  wise 
man  was  scheming  to  marry  the  king’s  daughter.  A  very  large  number  of  beliefs  of  this 
sort  have  no  bearing  on  the  problem  at  hand,  but  would  tend  to  use  up  valuable  mental 
resources  if  they  were  given  any  serious  consideration.  One  can  imagine  an  unsure  agent  who 
could  never  come  to  any  negative  conclusions  at  all,  because  he  would  keep  on  considering 
more  and  more  possibilities  for  solving  a  problem.  This  agent’s  reasoning  might  proceed 
as  follows:  I  can’t  tell  the  color  of  my  spot  by  looking  at  the  other  wise  men.  But  maybe 
there's  a  mirror  that  shows  my  face.  No,  there's  no  mirror.  But  maybe  my  brother  wrote 
the  color  on  a  slip  of  paper  and  handed  it  to  me.  No,  there's  no  slip  of  paper,  and  my 
brother's  in  Babylon.  But  maybe  ... 

McCarthy  (in  [27])  first  called  attention  to  the  problem  of  representing  what  is 
not  tbe  case  in  solving  puzzles.  In  the  Missionaries  and  Cannibals  Puzzle,  why  can’t  the 
missionaries  simply  use  the  bridge  downstream  to  get  across?  A  straightforward  logical 
presentation  of  the  puzzle  doesn’t  explicitly  exclude  the  existence  of  such  a  bridge.  And, 
if  it  did,  we  could  always  come  up  with  other  modes  of  transportation  that  had  not  been 
considered  beforehand  and  explicitly  excluded.  McCarthy  called  the  general  problem  of 
specifying  what  conditions  do  not  hold  in  a  puzzle  the  circumscription  problem.  By  analogy, 
we  call  the  problem  of  specifying  what  beliefs  an  agent  does  not  have,  or  does  not  use  in 
solving  a  given  task,  the  problem  of  circumscriptive  ignorance  (see  Konolige  [20]).  Without 
a  solution  to  this  representational  problem,  all  agents  will  be  modeled  as  unsure  agents  - 
never  able  to  reach  a  conclusion  about  what  they  don’t  believe,  even  though  it  is  obvious 
when  the  set  of  relevant  beliefs  is  circumscrihed. 

Of  course,  if  an  agent  can  circumscribe  his  beliefs.  It  is  possible  that  he  will  choose 
the  wrong  set  of  beliefs,  and  exclude  some  that  actually  are  relevant.  The  not-so-wise-man 
may  decide  that  the  beliefs  of  the  first  wise  man  are  not  germane  to  the  problem  of  figxiring 
out  his  own  spot’s  color.  Thus,  even  though  he  has  all  the  relevant  information,  and  even 
sufficiently  powerful  inference  rules  and  adequate  resources,  he  may  fail  to  come  to  a  correct 
conclusion  because  he  has  circumscribed  his  beliefs  in  the  wrong  way.  We  call  this  type  of 
incompleteness  relevance  incompleteness. 
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Within  a  model  of  belief  that  assumes  consequential  closure,  it  is  possible  to  represent 
circumscriptive  ignorance,  but  only  in  a  relatively  limited  fashion.  If  consequential  closure 
is  assumed,  one  can  state  that  an  agent  is  ignorant  of  some  fact  which  is  not  a  logical 
consequence  of  his  beliefs  (McCarthy  [25]  uses  this  technique  in  his  solution  to  the  Wise 
Man  Puzzle).  But  this  clearly  does  not  capture  the  complete  conditions  of  circumscriptive 
ignorance,  since  agents  are  often  ignorant  of  some  of  the  logical  consequences  of  their  beliefs, 
as  in  the  chess  scenario. 

Modeling  relevance  incompleteness  (or  having  the  third  wise  man  do  so)  is  impossible 
if  it  is  assumed  that  the  beliefs  of  agents  are  consequentially  complete.  One  simply  cannot 
partition  the  set  of  beliefs  into  those  that  are  either  relevant  or  not  to  a  given  problem; 
all  the  consequences  of  beliefs  are  believed.  If  we  try  to  state  the  conditions  of  relevance 
incompleteness  within  such  a  model,  we  can  arrive  at  a  contradiction,  where  a  proposition 
is  both  believed  (because  of  the  assumption  of  consequential  closure)  and  not  believed 
(because  of  the  condition  of  relevance  incompleteness). 
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3. 


The  Deduction  Model  of  Belief 


The  two  belief  representation  problems  can  be  solved  within  the  framework  of  a 
formal  model  of  belief  that  we  call  the  deduction  model.  In  this  section  we  define  the 
mode!;  in  the  next  we  introduce  a  logic  family  B  as  its  axiomatization. 

The  strategy  we  pursue  is  to  first  examine  the  way  typical  AI  robot  planning  systems 
(STRIPS  [9],  NOAH  [37],  WARPLAN  [42],  KAMP  [l],  etc.)  represent  and  reason  about 
the  world.  This  leads  to  the  identification  of  an  abstract  belief  subsystem  as  the  internal 
structure  responsible  for  the  beliefs  of  these  agents.  The  characteristics  of  belief  subsystems 
can  be  summarized  briefly  as  follows. 

1.  A  belief  subsystem  contains  a  Ibt  of  sentences  in  some  Internal  (“men¬ 
tal”)  language,  the  base  beliefs. 

2.  Agents  can  infer  consequences  of  their  beliefs  by  syntactic  manipula¬ 
tion  of  the  sentences  of  the  belief  subsystem. 

3.  The  derivation  of  consequences  of  beliefs  is  logically  incomplete,  be¬ 
cause  of  limitations  of  the  inferential  process. 

Having  identified  a  belief  subsystem  as  that  part  of  an  agent  responsible  for  beliefs, 
our  next  task  is  to  define  a  formal  mathematical  structure  that  models  it  accurately.  The 
decisions  to  be  made  here  involve  partic\ilar  choices  for  modeling  the  various  components 
of  a  belief  subsystem:  What  does  the  internal  language  look  like?  What  kind  of  inference 
process  derives  consequences  of  the  base  beliefs?  and  so  on.  The  formal  mathematical 
object  we  construct  according  to  these  criteria  is  called  a  deduction  structure.  Its  main 
components  are  a  set  of  sentences  in  some  logical  language  (corresponding  to  the  base  beliefs 
of  a  belief  subsystem)  and  a  set  of  deduction  rules  (corresponding  to  the  belief  inference 
rules)  that  may  be  logically  incomplete.  Because  we  choose  to  model  belief  subsystems 
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in  terms  of  lo^cal  (but  perhaps  incomplete)  deduction,  we  call  it  the  deduction  model  of 
belief. 


3.1.  Planning  and  Beliefs:  the  Belief  Subsystem  Abstraction 

A  robot  planning  system,  such  as  STRIPS,  must  represent  knowledge  about  the 
world  in  order  to  plan  actions  that  affect  the  world.  Of  course  it  is  not  possible  to  represent 
all  the  complexity  of  the  real  world,  so  the  planning  system  uses  some  abstraction  of 
properties  of  the  real  world  that  are  important  for  its  task;  e.g.,  it  might  assume  that  there 
are  objects  that  can  be  stacked  in  simple  ways  (the  blocks  world  domain).  The  state  of 
the  abstract  world  at  any  particular  point  in  time  has  been  called  a  situation  in  the  AI 
literature. 

In  general,  the  planning  system  will  have  only  incomplete  knowledge  of  a  situation. 
For  instance,  if  it  is  equipped  with  visual  sensors,  it  may  be  able  to  see  only  some  of  the 
objects  in  the  world.  What  this  means  is  that  the  system  has  to  be  able  to  represent  and 
reason  about  partial  descriptions  of  situations.  The  process  of  deriving  beliefs  is  a  symbol- 
manipulating  or  syntactic  operation  that  takes  as  input  sentences  of  the  formal  language, 
and  produces  new  sentences  as  output.  Let  us  call  any  new  sentences  derived  by  inferences 
the  inferable  sentences,  and  the  process  of  deriving  them  belief  inference. 

It  is  helpful  to  view  the  representation  and  deduction  of  facts  about  the  world  as 
a  separate  subsystem  within  the  planning  system;  we  call  it  the  belief  subsystem.  In  its 
simplest,  most  abstract  form,  the  belief  subsystem  comprises  a  list  of  sentences  about  a 
situation,  together  with  a  process  for  deriving  their  consequences.  It  is  integrated  with  other 
processes  in  the  planning  system,  especially  the  plan  derivation  process  that  searches  for 
sequences  of  actions  to  achieve  a  given  goal.  In  a  highly  schematic  form,  Figure  1  sketches 
the  belief  subsystem  and  its  interaction  with  other  processes  of  the  planning  system.  The 
belief  system  is  composed  of  the  base  beliefs,  together  with  the  belief  inference  process. 
Belief  inference  itself  can  be  decomposed  into  a  set  of  inference  rules  and  a  control  strategy 
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Figure  1.  Schematic  of  a  Belief  Subsystem 

that  determines  how  the  rules  are  to  be  applied  and  where  their  outputs  go  when  requests 
are  made  to  the  belief  subsystem. 

A  belief  subsystem  defines  an  agent’s  beliefs  by  the  action  of  the  inference  rules  on 
the  base  beliefs,  under  the  guidance  of  the  control  strateg}'.  Some,  but  not  necessarily  all, 
of  the  inferable  sentences  will  be  beliefs  of  the  agent;  which  inferable  sentences  are  actually 
beliefs  depends  on  the  details  of  the  control  strategy  and  the  resources  available  for  belief 
inference. 

There  are  two  types  of  requests  that  result  in  some  action  in  the  belief  subsystem. 
A  process  may  request  the  subsystem  to  add  or  delete  sentences  in  its  base  beliefs;  this 
happens,  for  example,  when  the  plan  derivation  process  decides  which  sentences  hold  in  a 
new  situation.  The  problem  of  updating  and  revising  beliefs  in  the  face  of  new  information 
is  a  complicated  research  issue  in  its  own  right,  and  we  do  not  address  it  here  (see  Doyle 
[7]  for  some  related  AI  research).  The  second  type  of  request  is  a  query  as  to  whether  a 
sentence  is  a  belief  or  not.  This  query  causes  the  control  strategy  to  try  to  infer,  using 
the  its  rules,  that  the  sentence  is  follows  from  the  base  beliefs.  It  Is  this  process  of  belief 
querying  that  we  model  in  this  paper. 

The  above  description  of  the  operation  of  a  belief  subsystem  is  meant  to  convey 
the  idea  that  in  most  formal  planning  systems  there  is  a  tight  interaction  between  belief 
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subsystems  and  planning.  Different  systems  may  deviate  from  the  described  pattern  to  a 
greater  or  lesser  extent.  In  some  systems,  the  representation  of  facts  may  be  so  limited, 
and  that  of  actions  so  e.xplicit,  as  to  almost  obviate  the  need  for  belief  deduction  per  se 
(as  in  some  versions  of  STRIPS).  In  others,  deduction  may  be  used  to  calculate  all  the 
effects  of  an  action  by  expanding  the  representation  to  include  situations  as  objects  (as  in 
WARPLAN).  Here  it  is  hard  to  make  a  clean  separation  between  deductions  performed 
for  the  purpose  of  deriving  consequences  of  beliefs  and  those  that  establish  the  initial  set 
of  facts  about  a  new  situation.  However,  it  is  still  conceptually  useful  to  regard  the  belief 
subsystem  as  a  separate  structure  and  belief  derivation  as  a  separate  process  within  the 
planning  system. 


3.2.  A  Formal  Model  of  Belief 


The  formal  mathematical  object  we  use  to  model  belief  subsystems  is  called  a  deduc¬ 
tion  structure.  A  deduction  structure  is  a  tuple  consisting  of  two  sets  and  will  be  written 
as  {B,  ^).  The  set  d  is  a  set  of  sentences  in  some  language  L;  It  corresponds  to  the  base 
beliefs  of  a  belief  subsystem  and  its  members  are  referred  to  as  the  base  sentences  of  the 
deduction  structure.  ^  is  a  set  of  deduction  rules  for  L;  these  correspond  to  the  inference 
rules  of  a  belief  subsystem.  We  demand  that  deduction  structures  satisfy  the  following  four 
conditions. 


Language  Property. 
Deduction  Property, 


Closure  Property. 


Recursion  Property. 


The  language  of  a  deduction  structure  is  a  logical  lau' 
guage. 

The  rules  of  a  deduction  structure  are  logical  deduc' 
tion  rules.  These  rules  are  sound,  effectively  com¬ 
putable,  and  have  bounded  input. 

The  belief  set  of  a  deduction  structure  is  the  least  set 
that  includes  the  base  sentences  and  is  closed  under 
derivations  by  the  deduction  rules. 

The  intended  model  of  deduction  structure  sentences 
involving  belief  is  the  belief  set  of  another  deduction 
structure. 


We  discuss  each  of  these  properties  briefly  below.  For  the  interested  reader,  a  more 
thorough  treatment  of  the  mathematical  properties  of  deduction  structures  is  given  in  the 
next  subsection. 


14 


About  the  only  condition  we  require  of  L  is  that  it  be  a  logical  language.  Logical 
languages  are  distinguished  by  having  a  constructable  set  of  syntactic  objects,  the  sentences 
of  the  language,  together  with  an  interpretatioD  method  (a  means  of  assigning  true  or  false 
to  every  sentence  with  respect  to  a  given  state  of  affairs). 

J?  is  a  set  of  deduction  rules  that  operate  on  sentences  of  L.  We  will  leave  unspeciHed 
the  exact  form  of  the  deduction  rules  Z,  but  we  do  insist  that  they  operate  in  the  normal 
manner  of  deduction  rules  in  some  proof-theoretic  framework.  This  means  that  there  is  the 
concept  of  a  derivation  of  a  sentence,  which  is  a  structure  built  from  effective  applications 
of  the  rules  Z.  irp  is  derivable  from  the  set  of  sentences  f  in  this  manner,  we  write  F  p, 
where  g- 15  is  a  derivation  operator  for  the  rules  Z.  For  example,  in  terms  of  Hilbert  systems 
(as  defined  in  Kleene  [18]),  Z  would  be  a  set  of  logical  axioms  (zero-premise  rules)  together 
with  modus  ponens  (a  two-premise  rule).  A  sentence  p  would  be  derivable  from  the  premise 
sentences  B  =  {61,621  •  •  •}  if  there  were  a  Hilbert  proof  of  (61  a 62  a.  . ,)  sp,  using  the  logical 
axioms  and  modus  ponens. 

A  deduction  structures  models  beliefs  by  its  belief  set,  which  we  define  as  follows. 
Definition  3.1. 

hel{{B,Z))=if{p\B^!lp}  . 

The  belief  set  is  composed  of  all  sentences  that  are  derivable  from  the  base  set  B  with 
the  rules  Z.  The  derivation  operator  thus  corresponds  to  the  belief  inference  process  of 
belief  subsystems. 

For  several  technical  reasons,  we  restrict  the  derivation  operators  allowed  in  deduc¬ 
tion  structures  to  those  that  satisfy  a  deductive  closure  condition.  One  consequence  of  this 
assumption  is  that  the  belief  set  itself  obeys  a  closure  property:  if  the  sentence  p  can  be 
derived  from  the  sentences  in  a  belief  set,  then  it  too  must  be  present  in  the  belief  set.  By 
making  the  assumption  of  deductive  closure,  the  task  of  formalizing  and  reasoning  about 
deduction  structures  is  greatly  simplified. 

It  is  important  to  note  that  deductive  closure  does  not  entail  consequential  closure 
for  belief  derivation:  a  set  of  sentences  closed  under  logically  incomplete  deduction  rules 
need  not  contain  all  logical  consequences  of  the  set.  This  is  an  important  property  of 
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deduction  structures,  and  it  enables  them  to  capture  the  behavior  of  belief  subsystems 
with  resource-bounded  control  strategies. 

Finally,  we  single  out  certain  sentences  of  the  deduction  structure  for  special  treat¬ 
ment,  namely  the  ones  that  themselves  refer  to  the  beliefs  of  agents.  In  discussing  the 
not-so-wise-man  problem  in  the  previous  section,  we  mentioned  that  one  of  the  key  tests  of 
a  belief  model  is  its  ability  to  handle  nested  beliefs  by  assuming  that  agents  use  the  model  in 
representing  other  agents’  beliefs;  a  belief  model  that  has  this  characteristic  is  said  to  have 
the  recursion  property.  In  terms  of  deduction  structures,  the  recursion  property  implies 
that  the  sentences  of  the  internal  language  L  that  are  about  beliefs  should  have  another 
deduction  structure  as  their  intended  interpretation. 

3.3.  Properties  of  Deduction  Structures 

In  this  subsection  we  treat  the  mathematical  properties  of  deduction  structures  in 
some  detail,  taking  care  to  show  how  they  can  model  the  behavior  of  belief  subsystems  of 
formal  Al  planning  systems. 

Language  Property 

One  restriction  we  plsice  on  the  language  of  deduction  structures  is  that  sentences 
of  the  language  have  a  well-defined  (i.e.,  truth-theoretic)  semantics.  Such  a  requirement 
seems  absolutely  necessary  if  we  are  going  to  talk  about  the  beliefs  of  an  agent  being  true 
of  the  actual  world,  or,  as  we  will  want  to  do  in  discussing  the  rationality  of  agents,  judge 
the  soundness  of  belief  deduction  rules.  Such  concepts  make  no  sense  in  the  absence  of  an 
interpretation  method  -  a  systematic  way  of  assigning  meanings  to  the  constructions  of  the 
language.  As  Moore  and  Hendrix  ([31],  parts  IV  and  especially  V)  note,  the  interpretation 
method  is  not  something  the  agent  carries  around  in  his  head;  a  belief  subsystem  is  just  a 
collection  of  sentences,  and  computational  processes  manipulate  the  sentences  themselves, 
not  their  meanings.  One  simply  cannot  put  the  referent  of  “Cicero”  into  a  robot’s  compu¬ 
tation  device,  even  if  he  (Cicero,  of  cotirse)  were  alive.  But  the  attribution  of  semantics  to 
sentences  is  necessary  if  an  outside  observer  is  to  analyze  the  nature  of  an  agent’s  beliefs. 
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How  well  do  actual  robot  belief  subsystems  lit  in  with  the  assumption  of  a  lo^cal 
language  of  belief?  AI  systems  xise  a  variety  of  representational  technologies;  chief  among 
these  are  frames,  scripts,  semantic  nets,  and  the  many  refinements  of  first-order  logic  (FOL), 
including  PROLOG  and  the  “procedurally  oriented”  logics  of  /^-PLANNER,  CONNIVER, 
QA4,  and  the  like.  The  representations  that  fall  into  the  fatter  category  inherit  their 
semantics  from  FOL.  despite  many  differences  in  the  syntactic  form  of  their  expressions. 
But  what  can  we  say  about  the  first  three?  In  surface  form  they  certainly  do  not  look 
anything  like  conventional  mathematical  logics;  furthermore,  their  designers  often  have 
not  provided  anything  but  an  informal  idea  of  what  the  meanings  of  expressions  in  tbe 
language  are.  When,  after  all,  is  a  pair  of  nodes  connected  by  a  directed  arc  true  of  the 
world?  As  Hayes  [11]  has  forcefully  argued,  the  lack  of  a  formal  semantics  is  a  big  drawback 
for  these  languages.  Fortunately,  on  further  examination  it  is  often  possible  to  provide  such 
a  semantics,  usually  by  transliterating  the  representation  into  a  first-order  language  (see 
Woods  [44]  and  Schubert  [39]  for  a  reconstruction  of  semantic  nets  in  FOL  terms,  and 
Brachman  [4]  for  a  similar  analysis  of  frames). 

In  discussing  human  belief,  several  philosophers  of  mind  have  argued  that  internal 
representations  that  count  as  beliefs  must  have  a  truth-value  semantics  (see  Fodor  [10], 
Field  [8],  and  Moore  and  Hendrix  [31]  for  a  discussion  of  the  many  intricate  arguments  on 
this  subject,  especially  pp.  48ff.  of  Field  and  part  V  of  Moore  and  Hendrix).  However, 
there  almost  certainly  is  a  lot  more  to  human  belief  than  can  be  bandied  adequately  within 
the  framework  of  a  logical  language.  For  example,  the  question  of  membership  in  the  belief 
set  of  a  deduction  structure  is  strictly  two-valued:  a  sentence  is  either  a  member  of  the 
belief  set  of  a  deduction  structure,  or  it  is  not.  If  it  is,  then  the  assumed  interpretation  is 
that  the  agent  believes  that  sentence  to  be  true  of  the  world.  Deduction  structures  thus  do 
not  support  tbe  notion  of  imcertain  beliefs  directly,  as  they  might  do  if  fuzzy  or  uncertain 
membership  in  the  belief  set  were  an  inherent  part  of  their  structure.* 


One  further  requirement  is  that  L  contain  expressions  referring  to  the  beliels  of 
agents.  Generally  we  will  take  this  to  be  a  belief  operator  whose  argument  is  an  expression 
in  L. 

^  However,  uncertain  beliefs  could  always  be  introduced  into  deduction  structures  in  an  indirect  manner  by 
letting  L  contain  statements  about  uncertainty,  e.g.,  statements  of  the  form  P  h  true  with  probability  1/2. 
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Finally,  it  is  often  the  case  that  we  will  want  to  freeze  the  language  of  deduction 
structures  in  order  to  study  their  properties  at  a  finer  level  of  detail,  e.g.,  when  looking  at 
the  behavior  of  nested  beliefs  in  general  or  when  giving  the  particulars  of  the  solution  to 
a  representational  problem.  It  is  convenient  to  think  of  the  language  as  being  a  parameter 
of  the  formal  model.  For  every  logical  language  L,  thiere  is  a  class  of  deduction  structures 
D(L,p)  whose  base  sets  are  sentences  of  the  language  L  (the  parameter  p  will  be  explained 
in  discussing  the  recursion  property  below). 

Deduction  Property 

Rules  for  deduction  structures  are  rules  of  inference  with  the  following  restrictions; 

1.  The  rule  is  an  effectively  computable  function  of  sentences  of  L. 

2.  The  number  of  input  sentences  is  boundedly  finite. 

3.  The  conclusion  is  sound  with  respect  to  the  semantics  of  L. 

These  restrictions  are  those  normally  associated  with  deduction  rules  for  classical  logic, 
although,  strictly  speaking,  deduction  rules  need  not  be  sound,  if  one  is  just  interested  in 
proof-theoretic  properties  of  a  logic  without  regards  to  semantics. 

The  fact  that  belief  deduction  rules  are  effectively  computable  functions  means  that 
they  can  be  very  complicated  indeed.  Mathematical  logicians  are  interested  in  logics  with 
simple  deduction  rules  (such  as  Hilbert  systems)  because  it  is  easy  to  analyze  the  proof- 
theoretic  structure  of  such  systems.  However,  for  the  purpose  of  deriving  proof  methods 
for  commonsense  reasoning  in  AI,  it  is  often  better  to  sacrifice  simplicity  for  computational 
efficiency.  For  example,  Robinson’s  resolution  rule  [36],  which  employs  a  matching  process 
called  unification,  is  a  complicated  rule  that  has  been  widely  employed  in  Al  theorem- 
proving.  Another  important  technique  is  Weyhrauch’s  semantic  attachment  [*13],  a.general 
framework  for  viewing  the  results  of  computation  as  deductions.  In  this  paper,  we  will 
exploit  complicated  rules  that  perform  deductions  that  are  relatively  “large”  with  respect 
to  the  grain  size  of  the  predicates,  particularly  in  solving  the  chess  problem  of  Section  2. 
Although  these  “large”  deductions  could  be  broken  down  into  smaller  steps,  it  is  computa¬ 
tionally  and  conceptually  easier  to  view  them  as  single  deductions. 

We  call  an  inference  rule  provincial  if  the  number  of  its  input  sentences  is  boundedly 
finite;  deduction  rules  are  always  provincial.  We  thus  do  not  allow  Inferences  about  beliefs 
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that  take  an  infinite  number  of  premises.  For  example,  the  following  rule  of  Carnap's  is  not 
a  valid  rule  of  belief  deduction;  if  for  every  individual  a:  F(a)  is  a  theorem,  then  Vz.F(z) 
is  a  theorem.^  Provincial  inference  rules  have  the  following  interesting  property:  if  a  is 
a  consequence  of  a  set  of  sentences  S  by  the  rule,  then  it  is  also  a  consequence  of  any 
larger  set  S'  D  S.  To  see  that  this  must  be  so,  consider  that,  if  a  can  be  derived  by  the 
application  of  provincial  rules  on  the  set  of  sentences  S,  and  S'  contains  S,  then  the  same 
derivation  can  be  performed  by  using  S'.  Rules  that  adhere  to  this  property  are  called 
jnonotoDic.  Technically,  monotonicity  is  convenient  because  it  means  we  can  reason  about 
what  an  agent  believes  on  the  basis  of  partial  knowledge  about  his  beliefs.  A  derivation 
made  using  a  subset  of  his  beliefs  cannot  be  retracted  in  the  face  of  further  information 
about  his  beliefs. 

Several  types  of  nonmonotonic  (and  unsound)  reasoning  have  been  of  interest  to  the 
AI  community,  specifically 

Belief  revision:  the  beliefs  of  an  agent  are  updated  to  be  consis¬ 

tent  with  new  information  (e.g.,  Doyle  [7]). 

Default  reasoning:  an  agent  “jumps  to  a  conclusion'’  about  the  way 

the  world  is  (e.g.,  McCarthy  [27],  Reiter  [35]). 

Autoepistemic  reasoning;  an  agent  comes  to  a  conclusion  about  the  world 

based  on  his  knowledge  of  his  own  beliefs  (e.g., 

Collins  et  aj.  [6],  Moore  [30]). 

We  are  e.xplicitly  not  trying  to  arrive  at  a  theory  of  these  forms  of  reasoning.  Indeed, 
it  is  helpful  here  to  make  the  distinction  that  Israel  (in  [16])  advocates  between  inference 
or  reasoning  in  general  (which  may  have  nonmonotonic  properties)  and  the  straightforward 
deduction  of  logical  consequences  from  a  set  of  initial  beliefs.  It  is  the  latter  concept  only 
that  is  treated  in  this  paper. 

If  we  wish  to  accommodate  some  nonmonotonic  theory  formally  within  the  frame¬ 
work  of  the  deduction  model,  then  we  can  view  its  inferences  as  deduction  rules  operating 
on  deduction  structure  theories  as  a  syntactic  whole.  McCarthy  [27]  exploits  this  approach 
to  formalize  a  certain  type  of  useful  default  inference,  which  he  calls  c/rcumscrfption  (see 
the  description  of  the  not-so-wise-man  problem  in  Section  2).  In  defining  the  logic  B,  we 

^  I  am  indebted  to  David  Israel  for  pointing  out  this  example. 
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will  show  how  to  formalize  circumscriptive  ignorance,  a  type  of  nonmonototiic  inference,  in 
this  manner. 


Deduction  rules  for  belief  subsystems  must  also  be  sound.  A  sound  deduction  rule 
is  one  for  which,  if  the  premises  are  true  in  an  interpretation,  then  the  conclusion  will  be 
also  (see  Kleene  [18]).  Informally,  one  would  say  that  sound  deduction  rules  never  deduce 
false  conditions  from  true  premises.  Modus  ponens  is  an  example  of  such  a  rule:  if  p  and 
p  3  7  are  true,  then  q  must  also  be. 

Soundness  of  inference  is  an  important  property  for  robot  agents  in  deriving  conse¬ 
quences  of  their  beliefs.  We  would  not  want  a  robot  who  believed  the  two  sentences 

All  men  are  mortal. 

Socrates  is  a  man. 

to  then  deduce  (and  hence  believe)  the  sentence 
(3.2)  Socrates  is  not  mortay. 

Soundness  is  not  a  critical  assumption  for  the  deduction  model,  since  none  of  the 
major  technical  results  depend  on  it.  In  some  cases  we  may  wish  to  relax  it,  for  example,  in 
modeling  the  behavior  of  human  syllogistic  reasoning,  which  is  often  unsound  (see  Johnson- 
Laird  [17]). 


To  sum  up:  deduction  structures  are  restricted  to  using  inference  rules  which  are 
provincial,  sound,  and  effectively  computable.  Several  interesting  types  of  reasoning,  such 
as  reasoning  about  defaults  or  one’s  own  beliefs,  cannot  be  modeled  directly  as  deduction 
rules  over  sentences.  However,  they  can  be  incorporated  into  the  deduction  model  if  the 
input  to  the  rules  is  taken  to  be  the  deduction  structure  as  a  whole. 

Closure  Property 

The  closure  property  states  that  the  belief  set  of  a  deduction  structure  is  closed 
under  derivations.  Formally,  this  amoimts  to  the  following  conditions  on  the  belief  set. 

1.  BCbel((B,;S)). 

2.  If  r  C  bel((B,  Z))  and  F  B-p  P,  then  p  €  bel((B,  je». 
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Since  we  have  defined  the  belief  set  in  terms  of  the  belief  derivation  operator  B*;: 
(Definition  3.1),  we  can  reexpress  these  as  conditions  on  belief  derivation. 

(Reflexivity)  cr  a. 

(Closure)  If  T  /?  and  S  g-pj;)  cr,  then  T,  E  a. 

Reflexivity  guarantees  that  the  base  set  will  be  included  in  bel,  and  the  closure 
condition  establishes  closure  of  bel  under  derivation. 


The  chief  motivation  for  requiring  derivational  closure  is  that  it  simplifies  the  tech¬ 
nical  task  of  formalizing  the  deduction  model.  Consider  the  problem  of  formalizing  a  belief 
subsystem  that  has  a  complex  control  strategy  guiding  its  inferential  process.  To  do  this 
correctly,  one  must  write  axioms  that  not  only  describe  the  agendas,  proof  trees,  and  other 
data  structures  used  by  the  control  strategy,  but  also  describe  how  the  control  strategy 
guides  inference  rules  operating  on  these  structures.  Reasoning  about  the  inference  process 
involves  using  these  axioms  to  perform  deductions  that  simulate  the  belief  inference  process, 
a  highly  inefficient  procedure.  By  contrast,  the  assumption  of  derivational  closure  leads  to 
a  simple  formalization  of  deduction  structures  in  a  logic  B  that  incorporates  the  belief  in¬ 
ference  process  in  a  direct  way.  We  need  not  differentiate  between  a  belief  as  a  member  of 
the  base  set,  or  as  a  derived  sentence.  A  sentence  that  follows  from  any  members  of  the 
belief  set  is  itself  a  belief.  The  axiomatization  of  B  is  simplified,  since  we  need  only  have 
an  operator  whose  intended  interpretation  is  membership  in  the  belief  set.  In  Section  4,  we 
exploit  the  properties  of  closed  derivational  systems  to  exhibit  a  complete  axiomatization 
of  B,  using  techniques  that  are  manner  similar  to  the  procedural  attachment  methods  of 
Weyhrauch  [43]. 


The  closure  property  is  an  extremely  important  one,  and  we  should  examine  its 
repercussions  closely.  A  point  that  we  have  already  made  is  that  derivational  closure  is  not 
the  same  as  consequentitd  closure.  The  latter  refers  to  a  property  of  sets  of  sentences  based 
on  their  semantics:  every  logical  consequence  of  the  set  is  also  a  member  of  the  set.  The 
former  refers  to  the  syntactic  process  of  derivability;  if  the  rules  JZ  are  not  logically  complete, 
then  a  set  of  sentences  that  is  derivationally  closed  under  2  need  not  be  consequentially 
closed. 
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One  of  the  key  properties  of  belief  subsystems  that  we  wish  to  model  is  the  incom¬ 
pleteness  of  deriving  the  consequences  of  the  base  set  of  beliefs.  We  have  identified  three 
sources  of  incompleteness  in  belief  subsystems:  an  agent’s  belief  inference  rules  may  be  too 
weak  from  a  logical  standpoint,  or  he  may  decide  that  some  beliefs  are  irrelevant  to  a  query, 
or  his  control  strategy  may  perform  only  a  subset  of  the  inferences  possible  when  confronted 
with  resource  limitations.  The  assumption  of  derivational  closure  for  deduction  structures 
affects  their  ability  to  model  incomplete  control  strategies,  since  closure  demands  that  all 
possible  deductions  be  performed  in  deriving  the  belief  set. 

For  an  important  class  of  incomplete  control  strategies,  however,  there  is  a  cor¬ 
responding  complete  control  strategy  operating  on  a  different  set  of  inference  rules  that 
produces  the  same  beliefs  on  every  base  set.  The  criteria  that  defines  this  class  is  that  the 
control  strategy  use  only  a  local  cost  bound  in  deciding  to  drop  a  particular  line  of  infer¬ 
ence.  By  “local”  is  meant  that  the  control  strategy  will  always  pursue  a  line  of  inference 
to  a  certain  point,  without  regard  to  other  lines  of  inference  it  may  be  pursuing  in  parallel. 
Control  strategies  with  a  local  cost  bound  are  important  because  their  inferential  behavior 
is  predictable:  all  inferences  of  a  certain  sort  are  guaranteed  to  be  made. 

Deduction  structures  can  accurately  model  the  class  of  locally  boimded  incomplete 
control  strategies  by  using  an  appropriate  set  of  logically  incomplete  deduction  rules.  A. 
good  example  is  found  in  the  solution  to  the  chess  problem  in  Section  5.  The  agent’s  control 
strategy  applies  general  rules  about  chess  to  search  the  game  tree  to  only  a  limited  depth; 
this  is  modeled  in  a  deduction  structure  by  using  deduction  rules  that  work  only  above  a 
certain  depth  of  the  game  tree,  and  applying  them  exhaustively. 

In  belief  subsystems  whose  control  strategies  have  a  global  cost  bound,  the  concept 
of  belief  itself  is  complicated,  since  one  must  differentiate  between  base  beliefs  and  beliefs 
inferred  with  some  amount  of  effort.  Deduction  structures  are  only  an  approximate  model 
of  these  subsystems,  and  a  language  with  a  single  belief  operator  is  no  longer  sufficient  for 
their  axlomatization. 

Recursion  Property 

If  belief  subsystems  adhere  to  the  recursion  property,  then  agents  view  other  agents 
as  having  belief  subsystems  similar  to  their  own.  This  still  leaves  a  considerable  degree  of 
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flexibility  in  representing  nested  beliefs.  For  example,  an  agent  John  might  believe  that 
Sue's  internal  language  is  Li  and  that  she  has  a  set  of  deduction  rules  whereas  Kim’s 
internal  language  is  L2  and  her  deduction  rules  are  ^2-  addition,  John  might  believe 
that  Sue  believes  that  Kim’s  internal  language  is  £,3,  and  that  her  rules  are  ^3.  We  call 
the  description  of  a  belief  subsystem  at  some  level  of  nesting  a  view;  formally,  views  are 
sequences  of  agents’  names,  so  that  the  view  John,  Sue  is  Sue’s  belief  subsystem  as  John 
sees  it.  We  will  often  use  the  Greek  letter  v  to  stand  for  an  arbitrary  view,  and  lowercase 
Latin  letters  (i,  j,  etc.)  for  singleton  views,  which  are  agents’  actual  belief  subsystems. 
Since  the  formal  objects  of  the  deduction  model  are  deduction  structures,  these  will  be 
indexed  by  views  when  appropriate.  For  example,  the  djoiin,Sue  ^  deduction  structure 
modeling  the  view  John,  Sue. 

Obviously,  some  fairly  complicated  and  confusing  situations  might  be  described,  with 
agents  believing  that  other  agents  have  belief  subsystems  of  varying  capabilities.  Some  of 
these  scenarios  would  be  useful  in  representing  situations  that  are  of  interest  to  A1  systems; 
e.g.,  an  expert  system  tutoring  a  novice  in  some  domain  would  need  a  representation  of  the 
novice’s  deductive  capabilities  that  would  initially  be  less  powerful  and  complete  than  its 
own,  and  could  be  modiRed  as  the  novice  learned  about  the  domain. 

We  model  the  recursion  property  of  belief  subsystems  within  the  framework  of  de¬ 
duction  structures  by  allowing  sentences  of  L  to  refer  to  the  beliefs  of  agents.  A  standard 
construct  is  to  have  a  belief  operator  in  L:  an  operator  whose  arguments  are  an  agent  S 
and  a  sentence  P,  and  whose  intended  meaning  is  that  5  believes  P.  According  to  the 
recursion  property,  this  means  that  the  belief  operator  must  have  a  deduction  structure  as 
its  interpretation.  Deduction  rules  that  apply  to  belief  operators  will  be  judged  soimd  if 
they  respect  this  interpretation.  For  example,  suppose  a  deduction  structure  dt,  has  a  rule 
stating  that  the  sentence  “John  believes  (f  can  be  concluded  from  the  premise  sentences 
“John  believes  p”  and  “John  believes  png”.  This  is  a  sound  rule  of  d„  if  modus  ponens 
is  believed  to  be  a  rule  of  John’s  belief  subsystem  as  viewed  from  the  view  v,  since  the 
presence  of  p  and  p  3  g  in  a  deduction  structure  with  modus  ponens  means  that  g  will  be 
derived. 

Several  simplifying  assumptions  are  implicit  in  the  use  of  deduction  structures  to 
model  the  nested  views  of  belief  subsystems.  The  language  L  contains  a  belief  operator 
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that  denotes  membership  m  a  belief  set  (its  intended  interpretation),  and  so  L  can  describe 
what  sentences  are  contained  in  an  agent’s  belief  set.  However,  there  is  no  provision  in  L 
for  talking  about  the  deduction  rules  an  agent  uses.  Instead,  these  nested-belief  rules  are 
implicitly  specified  by  the  rules  that  manipulate  sentences  with  belief  operators.  Consider 
the  example  from  the  previous  paragraph.  Let  us  suppose  that  we  are  modeling  Sue’s 
belief  subsystem  with  the  deduction  structure  ds^g.  Because  Sue  believes  that  John  uses 
modus  ponens,  a  sound  rule  of  inference  for  dgue  would  be  the  one  that  was  stated  above, 
namely,  the  sentence  “John  believes  g”  could  be  concluded  from  the  premise  sentences 
“John  believes  p”  and  “John  believes  p  3  g.”  All  of  the  rules  that  Sue  believes  John  uses 
are  modeled  in  this  way.  Similarly,  if,  in  Sue’s  opinion,  John  believes  that  Kim  uses  a 
certain  rule,  this  will  be  reflected  in  a  rule  of  John’s  deduction  structure  as  seen  by  Sue, 
which  in  turn  will  be  modeled  by  a  rule  in  dgue-  deduction  model  thus  assumes  that 
the  rules  for  each  view,  though  they  may  be  different,  are  a  fixed  parameter  of  the  model. 
We  introduce  the  function  p(v)  to  specify  deduction  rule  sets  for  each  view  i/-  thus,  for 
each  function  p  and  each  language  L,  there  is  a  class  of  deduction  structures  D(L,p)  that 
formalize  the  deduction  model.  If  the  rules  p  are  complete  with  respect  to  the  semantics  of 
L,  then  the  class  is  said  to  be  saturated,  and  is  written  Dg(L,p) 

A  final  simplification  that  is  not  inherent  in  the  deduction  model,  but  which  we 
introduce  here  solely  for  technical  convenience,  is  to  assume  that  all  deduction  structures 
in  ail  views  use  the  same  language  L.  There  are  situations  in  which  we  might  want  to  relax 
this  restriction,  it  makes  the  axiomatlzation  less  complex  in  dealing  with  the  problems  at 
hand. 
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4.  The  Logic  Family  B 


We  now  define  a  family  of  logics  B(L,  p)  for  stating  facts  and  reasoning  about  de¬ 
duction  structures.  This  family  is  parameterized  in  the  same  way  as  deduction  structures, 
namely  by  an  agents’  language  L  and  an  ensemble  of  deduction  structure  rules  p.  Each 
logic  of  the  family  is  an  axiomatization  of  the  deduction  structures  0(L,  p). 

The  language  of  B  includes  operators  for  stating  that  sentences  are  beliefs  of  an 
agent,  but  not  for  describing  deduction  rules  of  agents.  Thus  the  deduction  rules  are  a 
parameter  of  the  logic  family,  and  are  fixed  once  we  decide  to  use  a  particular  logic  of  the 
family.  The  ensemble  function  p  picks  out  a  set  of  rules  for  each  agent.  The  reason  we  chose 
to  make  the  deduction  rules  a  parameter  of  B  is  that  it  is  then  possible  to  find  efficient 
proof  methods  for  B.  One  of  the  interesting  features  of  B’s  axiomatization  is  that  agents’ 
rules  are  actually  present  as  a  subset  of  the  rules  of  B;  proofs  about  deduction  structures 
in  B  use  these  rules  directly  in  their  derivation. 

The  logic  of  B  is  framed  in  terms  of  a  modified  form  of  Gentzen  systems,  the  block 
tableau  systems  of  Hintikka.  Although  they  may  be  unfamiliar  to  some  readers,  block 
tableaux  are  easy  to  work  with  and  possess  some  natural  advantages  when  applied  to 
the  formalization  of  deduction  structures.  Unlike  Hilbert  systems,  which  contain  complex 
logical  axioms  and  a  single  rule  of  inference  in  the  propositional  case  (modus  ponens),  block 
tableau  systems  have  simple  axioms  and  a  rich  and  flexible  method  of  specifying  deduction 
rules.  We  e.xploit  this  capability  when  we  incorporate  deduction  structure  rules  into  B. 

In  this  section  we  first  present  a  brief  overview  of  block  tableaux.  Then  we  give  the 
postulates  of  the  family  B,  and  a  particularly  simple  subfamily  called  BK  that  will  be  used 
in  solving  the  problems.  By  way  of  example,  we  prove  some  theorems  of  BK. 
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4.1.  Black  Tableaux 


Most  of  this  section  will  comprise  a  review  for  those  readers  who  are  already  familiar 
with  tableaux  systems. 

The  Base  Language  Lq 

The  language  of  B  is  formed  from  a  base  language  Lq  that  does  not  contain  any 
operators  referring  to  beliefs.  Lq  is  taken  to  be  a  first-order  language  with  constant  terms. 
.'\n  interpretation  of  Lq  is  a  truth-value  assignment  to  all  sentences  (closed  formulas)  of 
Lq\  this  assignment  must  be  a  first-order  va/uation,  that  is,  it  must  respect  the  standard 
interpretation  of  the  universal  and  existential  quantifiers  as  well  as  the  Boolean  connectives. 

We  call  Lq  uniaterpreted  if  every  first-order  valuation  is  an  interpretation  of  Lq; 
partially  interpreted  if  some  proper  subset  of  the  first-order  valuations  are  interpretations 
of  Lq;  and  fully  interpreted  (or  simply  interpreted)  if  there  is  a  singleton  interpretation  of 
Lq.  a  sentence  of  Lq  is  valid  if  and  only  if  it  is  true  in  every  interpretation  of  Lq. 

We  use  lowercase  Latin  or  Greek  letters  (p,  q,  a,  etc.)  as  metavariables  that  stand 
for  sentences  of  Lq.  A  formula  of  Lq  that  possibly  contains  the  free  variable  x  will  be 
indicated  by  a[x);  the  formula  derived  by  substituting  the  constant  a  everywhere  for  x  is 
denoted  by  a(xla).  Uppercase  Greek  letters  (T  ^  =df  s^c.) 

stand  for  finite  sets  of  sentences  of  Lq.  By  p,  T  we  mean  the  set  (p)  IJ  F.  We  also  introduce 
the  abbreviation  -iF  =(if 

5equents 

Sequents  are  the  main  formal  object  of  block  tableaux  systems. 

Definition  4.1.  a  sequent  /s  an  ordered  pair  of  finite  sets  of  sentences, 

(F,  A).  This  sequent  will  also  be  written  as  V  A,  and  read  as  “A  follows 
from  F.” 

A  sequent  F  ^  A  is  true  in  an  interpretation  of  its  component  sentences  iff 
one  of  7j  is  false,  or  one  of  is  true.  A  sequent  is  valid  iff  it  is  true  under 
all  interpretations,  and  satisfiable  iff  it  is  true  In  at  feast  one  interpretation. 
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From  the  defmition  of  truth  for  a  sequent,  it  should  be  clear  that  a  sequent  F  =>  :! 
is  true  in  an  interpretation  just  in  case  the  sentence  {7]  a  72  ^  •  •  •)  =*  {b\w  8ow . . .)  is  tnie  in 
that  interpretation.  Thus,  in  a  given  interpretation  a  true  sequent  can  be  taken  as  asserting 
that  the  conjunction  of  7’s  materially  implies  the  disjunction  of  the  <f’s. 

We  allow  the  empty  set  ^  to  appear  on  either  side  of  a  sequent,  and  abbreviate 
ip^Aby^A,  r^flJbyr=>,  and  ^  by  =>.  By  the  above  definition,  =>  A  is  true 
(in  an  interpretation)  if  and  only  if  one  of  5,-  is  true,  F  =>  is  true  if  and  only  if  one  of  7,-  is 
false,  and  ^  is  never  true  in  any  interpretation. 

Block  Tableaux  for  Lq 

The  proof  method  we  adopt  is  similar  to  Gentzen’s  original  sequent  calculus,  but 
simpler  in  form.  It  is  called  the  method  of  block  tableaux,  and  was  orginated  by  Hintikka 
[13].  A  useful  reference  is  Smullyan  [40],  in  which  many  results  in  block  tableaiLX  and 
similar  systems  are  presented  in  a  unified  form. 

A  block  tableau  system  consists  of  axioms  and  rules  (collectively,  postulates)  whose 
formal  objects  are  sequents.  Block  tableau  rules  are  like  upside-down  inference  rules:  the 
conclusion  comes  first,  next  a  horizontal  line,  then  the  premises.  Block  tableaux  themselves 
are  derivations  whose  root  is  the  sequent  derived,  whose  branches  are  given  by  the  rules, 
and  whose  leaves  are  axioms.  Block  tableaux  look  much  like  upside-down  Gentzen  system 
trees.  (A  more  formal  definition  of  block  tableaux  is  given  below). 


We  consider  a  system  Tq  (see  Smullyan  [40],  pp.  105-109)  that  is  first-order  sound 
and  complete:  its  consequences  are  precisely  the  sentences  true  in  every  first-order  valua¬ 
tion. 

Definition  4.2.  The  system  To  has  the  following  postulates. 


Axioms. 

r,p  => 

A,p 

Conjunction  Rules. 

Ci: 

r,p  A  g  =>  A 
r,p,g=>  A 

C2: 

F  =>  A,  p  A  g 

F  =>  A,p  F  =>  A,g 
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Disjuaction  Rules. 


Di  : 

Do  : 

Implication  Rules.  Ii  : 

l2‘. 

Negation  Rules.  iVj  : 

N21 

Universal  Rules.  U\  : 

V2: 

E.\istential  Rules.  Ei  : 

E2: 


r  =>  q 

r  =>  A,p,q 

r,p  V  q  =>  A 
r,p  ^  A  r,q  =>  A 

r  ^  A,p::>  q 
r,p=>  A.q 
r,p3  q=>  A 
r=>A,p  r,q=>A 

r  ^  A,  -Ip 
r,p  =>  A 
r,  -Ip  =>  A 
r  =>  A,p 

r,Vx.o(a:)  =>  A 
r,a(i/o),Vx.a(x)  =>  A 
r  =>  Vx.a(x),  A 
r  =>  a(x/o), Vx.a(*),  A’ 

r  =>  gx.o(x),  A 

r  =>  a(x/a),3x.a(x),  A 
r,3x.a(x)  =►  A 
r,Q(x/(j),3x.Q(x),  =>  A’ 


where  a  has  not  appeared  in 
the  tableau 


where  a  has  not  appeared  in 
the  tableau 


Remarks.  Note  the  simple  form  of  the  axioms  and  the  symmetric  nature  of  the  inference 
rules  (actually,  each  rule  is  a  rule  schema,  since  F,  A,  p,  q,  and  a  stand  for  formulas  and 
sets  of  formulas  of  Lq).  There  Is  one  rule  that  deletes  each  logical  connective  on  either  side 
of  the  sequent.  For  example,  the  first  conjunction  rule  deletes  a  conjunction  on  the  left 
side  of  a  sequent  in  favor  of  the  two  conjoined  sentences;  informally,  it  can  be  read  as  “A 
follows  from  F  and  p  a  q  if  it  follows  from  F,  p,  and  q.”  It  is  easily  verified  that  each  rule  is 
sound  with  respect  to  first-order  valuations;  if  the  premises  are  true  in  an  interpretation, 
then  so  is  the  conclusion. 

Definition  4.3.  A  block  tableau  for  the  sequent  F  =s>  A  in  a  system  T  is  a 
tree  whose  nodes  are  sequents,  defined  inductively  as  follows. 

1.  F  ^  A  is  the  root  of  the  tree. 


2S 


2.  If  sequent  a  is  the  parent  node  of  daughters  3i...3„, 

then  - is  a  rule  of  T . 

3l ...3„ 

A  block  tableau  is  closed  if  all  its  leaves  are  axioms.  If  there  is  a  closed 
block  tableau  for  the  sequent  F  =*•  A,  then  this  sequent  is  a  theorem  of  the 
system  T  and  we  write  I-7  F  A. 

A  system  T'  is  called  a  subsystem  of  T  if  every  rule  of  7'  is  also  a  rule  of 
T.  If  some  subsystem  T'  of  T  has  exactly  the  same  theorems  as  T,  then  the 
rules  of  T not  appearing:  in  T'  are  said  to  be  eliminable  from  T,  or  admissible 
to  7'. 

Block  tableaux  are  similar  to  the  AND/OR  trees  commonly  encountered  in  AI 
theorem-proving  systems  (see  Nilsson  [32]).  Rules  Cn,  Do,  and  /2  cause  AND-splitting, 
while  a  choice  of  rules  to  apply  at  a  tableau  node  is  an  OR-spHt. 


Example.  Here  is  a  block  tableau  for  the  sequent  3z.  Bx  a  .4x,  Vi.  Cz  3  -<Bx  =>  3z.  Az  a 


iC7i. 


£^2 


3z.  Bx  A  .Ax,  Vz.  Ci  3  ~^Bx  3z.  Ax  a  -^Cx 
Be  a  Ae,  Vz.  Cz  =  ^Bx  =>  3z,  Ai  a  ->Cx 


El 


Cl 


Be  a  Ae,Ce  3  -iBe  =>  3z. Az  a  '^^Cx 
Be  a  Ae,  Ce  3  -iBe  =>  Ae  a  -<Ce 
Ae,  Be,  Ce  =>  ~iBe  =»  Ae  a  -iCe 


^2 


Ae,  Be,  “'Be  =>  Ae  a  -^Ce 


Ae,  Be : 


Be,  Ae  a  ->C7e 

X 


C2 


Ae,  Be  =>  Ce,  Ae  a  -iCe 


■Ni 


Ae,Be  =>  Ce,~'Ce 


Ae,  Be  =>  Ce,  Ae 

X 


Ae,  Be,  Ce  =>  Ce 

X 

The  sequent  to  be  proved  is  inserted  as  the  root  of  the  tree.  By  a  series  of  reductions 
based  on  the  rules  of  7o,  the  atoms  of  the  sequent’s  sentences  are  extracted  from  the  scope 


of  quantifiers  and  Boolean  operators.  Splitting  of  the  tree  occurs  at  the  rules  I2  and  Co; 
otherwise  the  reduction  produces  just  a  single  sequent  below  the  line.  If  a  tree  is  found 
where  the  sequents  at  all  the  leaves  are  axioms,  then  the  theorem  is  proved.  Note  chat 
the  logical  inferences  are  from  the  leaves  to  the  root  of  the  tree,  even  though  we  work 
backwards  in  forming  the  tree.  At  each  junction  of  the  tree,  the  parent  sequent  is  true  in 
an  interpretation  if  all  its  daughters  are  true  in  that  interpretation. 


An  important  connection  between  theoremhood  and  logical  consequence  for  sequent 
systems  is  the  following  soundness  theorem  for  tableaux. 
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Theorem  4.  l.  /f  r  =>  p  w  a  theorem  of  T  (where  p  is  a  single  sentence  of 
Lq),  and  all  the  rules  ofT  are  sound,  then  p  is  a  logical  consequence  ofT. 

Proof.  If  the  rules  of  T  are  sound,  then  every  theorem  of  T  is  valid.  By 
Definition  4.1,  this  means  that  in  every  interpretation  in  which  all  of  F  are 
true,  p  must  be  also.| 

4.2.  The  Language  of  B 

The  language  of  B  is  formed  from  a  first-order  base  language  Lq  by  adding  modal 
operators  for  belief  and  belief  circumscription.  We  call  this  language  L®.  It  is  convenient 
to  use  L®  also  as  the  agents’  language  L,  since  it  provides  a  representation  for  nested  beliefs 
as  required  by  the  recursion  property.  With  this  assumption,  we  can  parameterize  B  by 
the  base  language  Lq,  and  write  B(Lo,p)  for  the  logic  family. 

To  form  L®  from  a  base  language  Lq,  we  require  a  countable  set  of  agents  (^o.  -S’!, 

...). 

Definition  4.4.  A  sentence  of  L®  based  on  Lq  is  defined  inductively  by  the 
following  rules. 

1.  All  formation  rules  of  Lq  are  also  formation  rules  of  L®. 

2.  If  p  is  a  sentence,  then  [Sflp  is  a  sentence  for »  >  0. 

3.  If  p  is  a  sentence  and  T  is  a  finite  set  of  sentences,  then 
(Si  ;  r)p  is  a  sentence  for  i  >  1. 

An  ordinary  atom  of  L®  is  a  ground  atom  of  Lq;  a  belief  atom  is  a  sentence  of 
the  form  [5,jp,  and  a  circumscriptive  atom  is  one  of  the  form  (Si  :  r)p.  In  the  belief 
atom  (5^jp,  p  is  said  to  be  in  the  context  of  the  belief  operator.  Note  that  there  is  no 
quantification  into  the  contexts  of  belief  atoms,  since  the  argument  of  a  belief  operator  is 
always  a  closed  sentence.  L®  can  be  extended  to  include  quantification  into  belief  contexts; 
such  a  language  has  greater  representational  power  and  its  logic  qB  has  a  more  complex 
axiomatization.  The  interested  reader  is  referred  to  Konolige  [21]  for  a  description  of  qB. 
Here,  the  simpler  B  is  sufficient  for  an  analysis  of  the  problems. 

We  will  use  the  abbreviation  [5)r  =df 
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Interpret^tioas 


Interpretations  of  the  language  of  L®  are  formed  from  interpretations  of  its  base 
language  Lot  together  with  an  interpretation  of  belief  and  circumscriptive  atoms.  The 
intended  meaning  of  the  belief  atom  [Sj-jp  is  that  p  is  in  the  belief  subsystem  of  agent  5,-; 
informally,  we  would  say  “5,-  believes  p.”  Since  we  are  formalizing  belief  subsystems  by 
means  of  deduction  structures,  an  interpretation  of  the  belief  atoms  [5,]p  is  given  by  a 
deduction  structure  cl{.  [S,-]p  is  true  if  p  is  in  bel(£/,-).  the  belief  set  of  d;;  otherwise  it  is 
false. 


In  addition  to  representing  beliefs  of  individuals,  we  use  belief  atoms  to  represent 
common  beliefs.  A  common  belief  is  one  that  every  agent  believes,  and  every  agent  believes 
every  other  agent  believes,  and  so  on  to  arbitrary  depths  of  belief  nesting.  We  reserve  the 
name  Sq  for  a  fictional  agent  whose  beliefs  are  taken  to  be  common  among  all  agents.  The 
belief  atom  [So]p  means  that  p  is  a  common  belief.  In  terms  of  deduction  structures,  its 
intended  interpretation  is  that  p  and  [5olp  are  in  the  deduction  structure  d,-  of  every  agent 
S{,  i  >  0. 

McCarthy  (see,  for  example,  [25])  was  the  first  to  recognize  the  common  knowledge 
could  be  represented  by  the  use  of  a  fictitious  agent  FOOL  whose  knowledge  “any  fool” 
would  know.  He  used  a  possible* worlds  semantics  for  knowledge,  and  so  all  consequences 
of  common  knowledge  were  also  known.  The  representation  of  common  belief  presented 
here  uses  an  obviously  similar  approach;  it  differs  only  in  that  common  belief  rather  than 
common  knowledge  is  axiomatized  (common  beliefs  need  not  be  true),  and  in  having  a 
deduction  structure  semantics,  so  that  common  beliefs  need  not  be  closed  under  logical 
consequence. 


The  interpretation  of  circumscriptive  atoms  is  also  given  by  the  deduction  structure 
representing  an  agent’s  beliefs.  The  intended  meaning  of  (5,-  :  r)p  is  that  p  is  derivable 
from  r  in  the  deduction  structure  cf,*,  that  is,  F  P-  The  circumscription  operator 
elevates  the  belief  derivation  process  to  a  first-class  entity  of  the  language  (as  opposed  to 
belief  operators  [5,],  which  simply  state  that  certain  sentences  are  in  or  not  in  the  belief 
set). 
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While  it  may  not  be  apparent  at  first  glance,  the  circumscription  operator  is  a 
powerful  tool  for  representing  situations  of  delimited  knowledge.  For  example,  to  formally 
state  the  condition,  “the  only  facts  that  agent  S  knows  about  proposition  p  are  F,"  we 
could  use 

(4.1)  (S  :  F)p  =  [5]p  . 

This  assertion  states  that  S  believing  p  is  equivalent  to  5  being  able  to  derive  p  from  F.  The 
forward  implication  is  uninteresting,  since  it  just  says  that  p  is  derivable  from  F  by  agent 
S,  J.e.,  {5jF  3  [5]p.  The  reverse  implication  is  more  interesting,  since  it  states  p  cannot  be 
a  belief  of  S  unless  it  is  derivable  from  F.  This  reverse  implication  limits  the  information 
S  has  available  to  derive  p  to  the  sentences  F,  and  thus  gives  the  circumscriptive  content 
of  (4.1).  Note  that  there  is  no  way  to  formulate  the  reverse  implication  as  a  sentence  of  L® 
using  only  belief  operators. 


The  reader  should  note  carefully  that  the  semantics  of  differs  completely  from 
that  of  most  modal  languages,  in  which  the  argument  to  the  modal  operator  is  usually 
taken  to  denote  a  proposition  that  can  take  on  a  truth-value  in  a  possible  world.  By 
contrast,  arguments  to  modal  operators  in  the  language  of  B  denote  sentences  of  L,  namely 
themselves.  It  is  important  to  keep  this  distinction  in  mind  when  interpreting  the  modal 
operators  of  B. 

4.3.  A  Sequent  System  for  B 

The  deductive  process  that  underlies  the  deduction  model  is  characterized  in  very 
general  terms  by  deduction  structures  and  their  associated  belief  sets.  Until  now  we  have 
been  content  with  deliberate  vagueness  about  the  exact  nature  of  deduction  rules  and  the 
derivation  process.  As  stated  in  Section  3,  there  are  five  conditions  that  must  be  satisfied: 
the  deduction  rules  must  be  effective,  provincial,  and  sound,  and  the  derivations  reflexive 
and  closed  under  deduction.  Consider  a  deduction  structme  rfj  =  for  agent  S,*.  If 

we  let  the  process  of  belief  derivation  for  d  be  symbolized  by  these  conditions  are  as 
follows. 

(Effectiveness)  The  deduction  rules  p(i)  are  effectively  applicable. 
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(Provinciality)  The  number  of  input  sentences  to  each  rule  is  finite  and 
bounded. 

(Soundness)  If  T  ot,  then  a  is  a  logical  consequence  of  P. 

(Reflexivity)  a  a. 

(Closure)  If  P  ^  and  E  or,  then  P,  E  a. 

Suppose  we  are  given  beforehand  a  derivation  operator  satisfying  the  above 

conditions,  that  models  an  agent  5,-’s  belief  subsystem.  The  central  problem  in  the  for¬ 
mulation  of  8  is  to  find  tableau  rules  that  correctly  implement  the  meaning  of  the  belief 
operator  [5,]  and  the  circumscription  operator  (5,- :  P)  under 

Consider  first  the  sequent  [5,-]P  ^  [5,-]Qf.  Its  intended  meaning  is  that,  if  all  of  P  are 
in  5,’s  belief  set,  then  so  is  a.  The  only  possible  way  that  we  can  guarantee  this  condition 
is  if  a  is  derivable  from  P,  /.e.,  P  a.  If  this  were  not  the  case,  then  we  could  always 
construct  the  counterexample  d{  =(jf  (P,  p{i))  in  which  all  of  P  are  in  d,-,  but  a  is  not.  Thus 
we  can  relate  the  truth  of  a  sequent  involving  belief  operators  to  derivability  in  an  agent’s 
belief  subsystem.  This  relation  is  captured  by  the  inference  rule 

E,  [5,-]P  =>  ^ 

'  r  a 

A  is  called  the  attachment  rule,  because  it  derives  results  involving  the  belief  op¬ 
erator  by  attaching  sentences  about  belief  to  the  actual  derivation  process  of  an  agent. 
Remembering  that  the  premise  is  the  bottom  sequent  and  the  conclusion  the  top,  we  can 
read  A  informally  as  follows:  “If  a  is  a  deductive  consequence  of  P  in  S,-’s  belief  subsystem, 
then,  whenever  5,-  believes  P,  he  also  believes  a.” 

To  capture  the  notion  of  common  belief,  we  need  to  make  a  modification  to  the 
attachment  rule.  The  intended  meaning  of  the  common  belief  atom  [5o]q  is  that  both  q 
and  [5o]7  are  in  the  belief  subsystem  of  every  agent.  The  sequent  [5o]A,  [5,-]r  =>  [5,]a  will 
be  true  if  whenever  (5o]A,  A,  and  P  are  in  the  belief  set  of  d,-,  a  also  is.  By  reasoning  similar 
to  that  used  in  deriving  the  rule  A,  we  can  rephrase  this  in  terms  of  belief  derivation.  This 
yields  the  revised  attachment  rule  A^^. 

^CB  .  E,  [5o)A,  [S^JP  ^  [■?i]q,  a 
{5o]A,  A,  P  Of 
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In  ,  both  A  and  [5ojA  can  be  used  in  the  derivation  of  a.  Note  that  this  rule  is  applicable 
to  the  fictional  agent  5o.  Because  Sq’s  beliefs  are  intended  to  be  common  beliefs,  and  hence 
derivable  by  any  agent,  it  should  be  the  case  that  the  rules  p{0)  are  used  by  every  agent. 
We  thus  demand  that  p(0)  C  ;j(i)  for  every  i. 


We  can  find  tableau  rules  for  the  circumscription  operator  in  a  similar  manner.  The 
intended  semantics  of  this  operator  relates  directly  to  the  belief  derivation  process:  (5,- ;  r)p 
means  that  p  is  derivable  from  F  in  5,-’s  belief  subsystem,  I'.e.,  F  p.  In  writing  sequent 
rules,  there  are  two  cases  to  consider,  for  a  circumscriptive  atom  can  appear  on  the  right 
or  left  side  of  the  sequent  arrow.  We  thus  have  the  following  two  rules. 


C«rci  : 


Circ2  *. 


^  ^  {S{  t  r)p,  A 

r  p 

s,  {Sj :  Dp  =>  A 

ri-p(o  p 


The  second  circumscription  rule  is  the  one  that  is  used  to  show  circumscriptive  igno¬ 
rance.  It  states  that  if  p  is  not  derivable  from  a  set  of  sentences  F,  then  the  circumscriptive 
atom  (S,-  :  F)p  is  false.  Given  a  statement  of  the  form  4.1,  this  in  turn  would  imply  that 
S,-  was  ignorant  of  p. 


We  can  now  give  a  full  a.xiomatization  of  the  logic  family  B. 

DEFINITION  4.5.  The  system  B(Xo)P)  bas  the  following  postulates. 

1.  The  first-order  complete  rules  Tq. 

2.  The  rules  Circi,  and  Circj. 

3.  A  closed  derivation  process  for  each  agent  5,-,  such 
that  p(0)  C  p(i)  for  every  i. 

This  axiomatization  of  B  is  both  sound  and  complete  with  respect  to  its  deduction 
structure  semantics,  as  proven  in  Konolige  [21].  It  is  a  compact  formalization  of  the  de¬ 
duction  model  and  useful  for  theoretical  investigations,  but  we  do  not  use  it  very  much  as 
a  representational  formalism  because  of  the  general  nature  of  the  belief  deduction  process 
which  is  rather  opaque  to  further  analysis.  For  instance,  we  might  wish  to  look  at 
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the  subfamily  of  B  in  which  the  rules  of  p(i)  that  govern  nested  belief  are  as  strong  as 
vA.  In  order  to  explore  the  fine  structure  of  S,’s  belief  deduction  process,  or  to  formalize 
the  problems,  we  need  to  fix  the  nature  of  more  precisely.  The  rich  set  of  rules,  and 
the  flexibility  of  tableau  derivations,  make  tableau  systems  a  natural  choice  here.  In  the 
next  section  we  define  a  particularization  of  B,  the  logic  family  BK,  whose  belief  derivation 
process  is  defined  in  block  tableaux  terms. 


4.4.  The  Nonintrospective  Logic  Family  BK 


In  the  logic  family  BK,  the  belief  derivation  operator  is  defined  as  provability  in  a 
tableau  system. 

Definition  4.6.  A  sentence  a  is  BK-derivabie  from  premises  F  (F  J-r  a)  if 
and  only  if\-T  F  ^  a. 


We  need  to  show  that  tableau  system  derivability  as  just  defined  satisfies  the  five 
criteria  of  belief  derivation:  effectiveness,  provinciality,  soundness,  reflexivity  and  closure. 
Consider  a  sequent  system  T  made  up  of  sound  tableau  rules.  According  to  Theorem  4.1, 
the  theorem  I-7  F  =>  p  of  T  implies  that  p  is  a  logical  consequence  of  F,  so  we  are  assured 
that  |-  r  satisfies  the  soundness  criterion.  Provinciality  and  effectiveness  are  also  satisfied, 
since  the  theorems  of  7  are  built  by  using  effectively  computable  steps  that  operate  on 
a  bounded  number  of  sentences  at  each  step.  The  observant  reader  might  object  at  this 
point  that  tableau  rules  may  indeed  refer  to  an  unbounded  number  of  premise  sentences; 
e.g.,  any  of  the  rules  of  Tq  have  this  property,  since  F  and  A  can  stand  for  any  set  of 
sentences.  However,  each  rule  of  To  is  actually  a  rule  schema:  the  capital  Greek  letters 
are  metavariables  that  are  instantiated  with  a  boundediy  finite  set  of  sentences  to  define  a 
rule. 


The  closure  condition  is  fulfilled  by  a  special  subclass  of  sequent  systems,  namely 
those  for  which  the  following  rule,  Cut*,  is  admissible: 

ft  i.*  F,  E  =>  tt 

Cut  :  - - 2 - -  • 

F  ^  ^  E  ^  (1 

To  see  how  this  rule  guarantees  closure,  suppose  that  F  ^  ^  and  ^,E  =>  a  are  both 
theorems  of  a  sequent  system  T  for  which  Cut*  is  admissible.  Because  Cut*  is  admissible 
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and  both  of  its  premises  have  closed  tableaux,  the  conclusion  F,!!  =>  a  must  also  be  a 
theorem. 

Finally,  the  derivation  process  will  be  reflexive  (a  \-j  a)  if  we  include  the  following 
axiom  in  the  system  T: 

Id:  E,  or  =:►  a,  A  . 

Thus  we  only  allow  a  system  T  to  appear  in  a  deduction  structure  d{B,  T)  if  the 
system  is  sound,  Cut*  Is  an  admissible  rule  of  T,  and  Id  is  an  axiom  of  T. 


An  interesting  consequence  of  using  tableau  derivations  in  BK  is  that  the  attach¬ 
ment  rule  A.  can  now  be  expressed  wholly  in  terms  of  sequents,  eliminating  the  derivation 
operator.  To  see  how  this  comes  about,  consider  first  replacing  the  belief  operator  in  rule 
A  by  tableau  provability,  as  given  by  Definition  4.6.  This  yields 

K(»)r=>a 

where  r(»)  is  the  set  of  tableau  rules  used  by  agent  5;. 


Now  r  =>  at  is  true  precisely  if  there  is  a  closed  tableau  for  F  ^  a,  using  the 
rules  r(i).  Hence  we  should  be  able  to  eliminate  the  provability  symbol  if  we  add  the  rules 
r(i)  to  B  for  the  purpose  of  constructing  a  tableau  for  F  =s>  a.  In  order  to  keep  the  agents* 
rules  r(<)  from  being  confused  with  the  rules  of  B,  we  add  an  agent  index  to  sequents  to 
indicate  that  the  tableau  rules  to  be  use  are  for  a  particular  agent.  The  final  version  of  the 
attachment  rule  is 


AK: 


S,[5^lF^[5,-ltt,A 

r  =>i  a 


Agents’  rules  are  expressed  using  the  inde.xed  sequent  sign,  e.g.,  if  agent  S{  were  to  use  Ci, 
the  following  rule  would  be  added  to  B. 

F  =>i  A  q 


Cj  • 


F=>,-A,p  r=>iX<i 


Taking  the  recursion  property  of  belief  subsystems  seriously,  we  can  iterate  the 
process  just  described  for  the  attachment  rule.  Each  agent  treats  other  agents  as  having 
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a  set  of  tableau  rules.  In  formulating  BK,  there  will  be  a  tableau  rule  set  associated  with 
each  view  (views  are  discussed  in  relation  to  the  recursion  property  in  Section  3.3).  Let  us 
symbolize  the  set  of  tableau  rules  representing  the  view  v  by  r(t'), 

A  sequent  T  =>i/  A,  with  index  i',  is  a  statement  about  the  belief  subsystem  of  the 
view  u.  For  example,  if  r  =  Sue,  Kim,  the  sequent  F  =>t,  p  states  that  p  follows  from  F 
in  Sue's  view  of  Kim’s  belief  subsystem.  The  deduction  rules  t{u)  always  have  sequents 
indexed  by  i/  in  their  conclusions  (above  the  line).  This  assures  us  that  they  will  always  be 
used  as  rules  of  the  belief  subsystem  i/,  and  of  no  other. 


The  logic  BK  can  thus  be  parameterized  by  a  set  of  tableau  rules  for  each  view, 
and  we  write  BK(Z,o,r)  to  indicate  this.  If  the  sequent  F  A  is  a  theorem  of  the  logic 
BK(Lo,r),  it  asserts  that  the  sequent  F  ^  A  is  provable  in  the  view  i/.  We  write  this  as 
(“BKILo,’')  r  sequent  is  a  theorem  for  every  parameterization  of  BK,  we  write 

simply  f-  F  =>„  A.  Note  that  the  presence  of  the  index  on  the  sequent  means  that  we  do 
not  have  to  state  explicitly  that  the  set  of  rules  used  to  derive  the  theorem  were  those  of  the 
view  u.  Properties  of  the  the  actual  belief  subsystems  are  always  stated  using  uninde.xed 
sequent;  for  example,  to  show  formally  that  if  an  agent  believes  p,  then  he  believes  q,  we 
would  have  to  prove  that  the  sequent  [S,-|p  ^  [5i]g  is  a  theorem  of  BK. 

Postulates  o/‘BK(£oiJ') 

This  family  is  parameterized  by  a  base  language  Lq  and  tableau  rules  ^(^')  for  each 

view 

Definition  4.7.  The  system  BK(£o>  J")  «  given  by  the  following  postulates: 

1.  The  hrst»onJer  complete  rules  Tq. 

2.  The  attachment  rule 

.  S,[gQ]A,[5,-]r  ^  [5, -la,  A 

[5o]A,A,r^.-a 

3.  A  set  of  sound  sequent  rules  r(t/)  for  each  view  u  which 

contains  the  axiom  Id,  and  for  which  the  rule  Cut*  is 
admissible.  Also,  r(i/,  0)  C  for  all  views  i>  and 

agents  5;. 
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4.  The  drcumscn'ptioa  rules 


and 


CircKi  : 


CircK2  : 


S=»<5.-:r)p,A 
r  =>{  p 

2,  (Sf  :  r)p  =>  A 

/•r=>ip 


Remarks,  There  are  three  parts  to  the  system  BK(Z,o,  r).  The  Erst  part  b  a  set  of  rules 
that  perform  first-order  deductions  about  the  real  world.  These  rules  incorporate  the 
nonsubscripted  sequent  sign  (^). 

The  second  part  is  the  attachment  rule  AK^^,  together  with  a  set  of  rules  formaliz¬ 
ing  the  deductive  system  of  each  view.  These  rules  involve  the  sequent  sign  =>:/,  since  they 
talk  about  agents’  deductive  systems.  They  can  contain  rules  that  have  a  purely  nonmodal 
import  (e.g.,  rules  of  Tq),  as  well  as  rules  that  deal  with  belief  operators.  The  rule  Cut*, 
which  implements  the  closure  property  of  belief  sets,  must  be  an  admissible  rule  of  r(t'). 

The  rules  r(t/)  of  a  view  w  can  be  incomplete  in  several  ways.  They  may  be  first-order 
incomplete,  in  which  case  they  cannot  be  used  to  draw  all  the  consequences  of  sentences 
involving  nonmodal  operators  that  they  otherwise  might  {to  be  first-order  complete,  it  is 
sufficient  for  the  rules  Tq  to  be  admissible  in  a  view).  They  may  also  be  incomplete  with 
respect  to  the  semantics  of  sentences  involving  belief  operators.  To  be  complete  in  this 
respect,  a  sufficient  rule  would  be  A  view  for  which  this  rule  b  admissible  is 

called  recursively  complete.  If  every  view  of  a  logic  BK(Lo,  r)  b  recursively  and  first-order 
complete,  the  logic  is  called  saturated.  We  will  symbolize  the  subfamily  of  saturated  logics 
by  BK,. 

The  rule  AK^^  b  a  weak  version  of  the  attachment  rule  in  that  it  makes  no 
assumptions  about  the  beliefs  an  agent  may  have  of  his  own  beliefs.  For  e.\ample,  we  might 
argue  that,  if  an  agent  5  believes  a  proposition  P,  then  he  believes  that  he  believes  it.  All 
he  has  to  do  to  establish  this  is  query  his  belief  subsystem  with  the  question,  “Do  I  believe 
P?”  If  the  answer  comes  back  “yes,”  then  he  should  be  able  to  infer  that  he  does  indeed 
believe  P,  i.e.,  [5)[5]P  is  true  if  [5]P  is.  However,  as  far  as  rule  AK^^  is  concerned,  an 
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agent's  own  belief  subsystem  has  the  same  status  for  him  as  does  that  of  any  other  agent. 
In  particular,  allows  an  agent  to  have  false  and  incomplete  beliefs  about  his  own 

beliefs.  Other  version  of  with  stronger  assumptions  about  self-belief  are  possible 

(see  Section  6). 


The  third  part  consists  of  the  two  circumscription  rules.  The  provability  operator 
can  be  eliminated  from  CircKi,  but  not  from  CircK2.  In  order  to  show  that  p  does  not 
follow  from  F  for  5,-,  we  must  show  that  there  is  no  closed  tableau  for  F  =>,•  p.  One 
technique  that  we  use  in  solving  the  problems  is  the  following.  If  there  is  no  closed  tableau 
for  a  saturated  logic  of  BK,  there  is  no  closed  tableau  for  any  logic  of  BK.  Every  theorem 
of  saturated  BK  is  a  theorem  of  the  normal  modal  system  K4  (see  Section  6),  which  has  a 
decision  procedure  based  on  the  methods  of  Sato  (in  [38]).  Thus  if  a  sequent  is  not  provable 
in  K4,  it  is  not  provable  in  any  logic  of  BK. 

Some  Theorems  of  BK 

Theorem  4.2.  Let  p  be  derivable  from  F  in  the  view  i  of  BK(Lo7’')-  Then 

l“BK{Lo,r)  • 

Proof.  In  one  step,  using  rule  : 

I-S'iir  =>  [.gtlp 

F  =>tP 

X 

I 

Theorem  4.3.  Let  1/  be  a  recursively  complete  view  ofBK(Lo,r),  and  let  p 
be  derivable  from  F  in  the  view  u,  1.  Then 

^■BK(Lo.r)  l^ilP  • 

Proof  In  one  step,  using  rule  of  t{u): 

r  =^u,i  P 

X 
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Remarks.  These  two  theorems  show  that  BK  has  a  weakened  analog  of  the  necessitation 
rule  of  modal  logic  (if  ct  is  provable,  so  is  Da).  If  a  nonmodal  sentence  a  is  provable  in  the 
view  I  (/.e.,  l-BK(Io.r)^*  Theorem  4.2,  [5,-Jo!  is  provable  in  the  empty  view. 

Since  the  theorems  of  r(i)  are  assumed  to  be  sound,  a  is  a  tautology,  and  so  must  be  provable 
in  the  empty  view.^  Hence,  for  those  tautologies  provable  in  the  view  t,  necessitation  holds. 
Theorem  4.3  establishes  this  result  for  an  arbitrary  view  in  which  .r4  is  an  admissible  rule. 
Depending  on  the  exact  nature  of  the  rule  sets  r,  necessitation  will  hold  for  some  subset  of 
the  provable  sentences  of  a  particular  logic  BK(I>o,  r). 


THEOREM  4.4.  jf-  [S,-]p  =>  p 

Proof.  If  p  is  a  primitive  sentence,  then  there  is  no  applicable  tableaux  rule, 
and  hence  no  closed  tableaux  for  the  sequent.  | 


Remarks.  The  familiar  modal  logic  principle  Dp  =  p  (if  p  is  necessary,  then  p  is  true)  is 
not  a  theorem  of  BK,  since  beliefs  need  not  be  true. 


T  HEOREM  4.5.  [5,-lp  =>  (5,-]  [5f]p 

Proof.  The  only  applicable  rule  is  : 

[■?,•]?  =>  [■g.-l  [■?«•]? 

P  =>i-  [-yilP 

According  to  the  semantics  of  the  deduction  model,  the  sequent  p  =>,•  [5,-]p 
is  not  valid:  just  because  a  sentence  p  is  true  does  not  mean  that  an  agent 
Si  believes  it.  Hence,  there  cannot  be  any  set  of  sound  tableau  rules  for  r(») 
that  causes  p  =>,■  [5,-]p  to  close.  | 

*  Care  must  be  taken  in  restricting  a  to  nonmodal  sentences,  since  the  semantics  of  modal  operators  can 
change  from  one  view  to  another  (see  the  discussion  of  the  recursion  property  in  Section  3.3).  John  may 
believe  perfectly  well  that  Sue’s  belief  subsystem  can  prove  a  certain  fact,  whereas  in  actuality  her  inference 
rules  are  too  weak. 
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Theorem  4.6. 


Proof.  We  can  apply  either  N2  or  .  If  we  apply  the  latter,  we  obtain 

=**.•  -'[5i]p 

deduction  model,  since  it  would  require  that  no  agent  believe  any  sentence. 
Hence  there  can  be  no  set  of  sound  tableau  rules  r(j)  that  derives  it. 

If  we  apply  N2  first,  we  obtain 

-■[■?<>  =>  [5,]-i[5,-]p 
^  =>l5i]p,[5,-b[5.-lp  • 

There  are  now  two  ways  to  apply  .  In  one  application,  we  generate 

the  sequent  ~'[5,-]p,  which  cannot  close.  In  the  other,  we  generate  p, 
which  again  cannot  be  derived  by  any  set  of  sound  tableau  rules. | 


Remarks.  These  theorems  show  that  no  logic  of  BK  sanctions  inferences  about  self-beliefs. 
If  an  agent  believes  p,  it  does  not  follow  that  his  model  of  his  own  beliefs  includes  p;  this 
is  the  import  of  Theorem  4.5.  Similarly,  if  he  does  not  believe  p,  he  also  may  not  have 
knowledge  of  this  fact,  as  shown  by  Theorem  4.6. 


Theorem  4.7. 

h  (5q1p  [‘^oll'^ojp 


Proof. 

.j^cB  I^oIp  =»  [^oK-gplp 

[■Sbjp.p  =>,•  [5o]p 

X 

I 


Remarks.  We  have  proven  a  simple  fact  about  common  beliefs:  if  p  is  a  common  belief, 
it  is  a  common  belief  that  this  is  so. 


For  the  circumscriptive  ignorance  part  of  BK,  it  is  an  interesting  exercise  to  show 

that 
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(4.2)  {Si  :  Dp  ^  [Si]r  ^  [5.-]p 


holds,  but  the  converse  doesn’t.  That  is,  it  p  follows  from  F  for  agent  5,-,  it  must  be  the 
case  that  believing  F  entails  believing  p;  on  the  other  hand,  it  may  be  that  every  time  an 
agent  has  F  in  his  base  set  he  also  has  p,  which  would  satisfy  (Sj-]F  s  [5j]p  without  having 
p  derivable  from  F. 

Theorem  4.8.  \-  {S^ F)p  =>  [5,-jF  =  [5{]p 

Proof.  We  have  the  following  two  tableaux  for  this  sentence. 

(5f:F}p=>[5,-lF^(5,-lp 

<5.-iF)p[5,-)r^[g,-]p 

F^fP 


A 

Cirti 


{5,-:F)p=>[5.]F^(5,]p 
{Si :  F)p[5^]F 

/■r^.P 


Either  p  is  derivable  from  F  using  the  rules  r(i),  or  it  isn’t.  In  either  case 
one  of  these  tableaux  closes.  | 


E.xampie.  we  give  an  example  of  the  use  of  the  circumscription  rules  to  show  ignorance. 
Suppose  the  agent  Sue  believes  only  the  sentences  P  and  P  3  (?  in  a  situation;  we  want  to 
show  that  she  doesn’t  believe  R.  Thus  we  want  to  prove  the  sequent  (Sue  :  P,  P  ^  Q)R  = 
[Sue]P  ^  -'(SueJP. 

^  •  (Sue  :  P,P 's  Q)R  =  [Sue]P  ^  -«[Sue]P 

}  {Sue  :P,PoQ)R3  [Suej/g  ,  [Sue\R  =>  {Sue  :P,P=>Q)R^  -.[SuejP 
(Sue  ;  P,  P  3  Q)R  ^  -'[Sue]P  ^  ^  [Sue]P,  ->[Sue]P 

f  P,P=><9  =>Sue  ^  ^  [Sue]P^  [Sue|P 

X 

If  the  rules  r(Sue)  are  sound,  there  is  no  closed  tableau  for  P,P  -n  Q  R,  and  so  both 
branches  of  the  tableau  close.  Note  that  only  the  reverse  implication  half  of  the  equivalence 
was  needed. 
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5.  The  Problems  Revisited 


Using  the  logic  BK,  we  present  formal  solutions  to  the  two  representational  problems 
posed  at  the  beginning  of  this  section.  In  each  case  we  have  tried  to  avoid  solutions  that  are 
trivial  in  the  sense  that  they  solve  the  representational  problem,  but  only  at  the  expense 
of  excluding  types  of  reasoning  that  might  be  expected  to  occur.  For  example,  in  the  chess 
problem  it  would  be  an  adequate  but  unrealistic  solution  to  credit  each  player  with  no 
deduction  rules  at  all.  Instead,  we  try  to  find  rules  that  allow  a  resource-limited  amount 
of  reasoning  about  the  game  to  take  place. 

The  Chess  Problem 

To  approach  this  problem,  we  need  to  represent  the  game  in  a  first-order  language. 
Because  the  ontology  of  chess  involves  rather  complicated  objects  (pieces,  board  positions, 
moves,  histories  of  moves)  we  will  not  give  a  complete  formalization,  but  rather  sketch  in 
outline  how  this  might  he  done. 

We  use  a  multisorted  first-order  language  Lc  for  the  base  language  Lq.  The  key  sorts 
will  be  those  for  players  (5t£;  or  S^),  moves,  and  boards.  The  particular  structure  of  the  sort 
terms  is  not  important  for  the  solution  of  this  problem,  but  they  should  have  the  following 
information.  A  board  contains  the  position  of  all  pieces,  and  a  history  of  the  moves  that 
were  made  to  get  to  that  position.  This  is  important  because  we  want  to  be  able  to  find 
all  legal  moves  from  a  given  position;  to  do  this,  we  have  to  have  the  sequence  of  moves 
leading  up  to  the  position,  since  legal  moves  can  be  defined  only  in  terms  of  this  sequence. 
For  example,  castling  can  only  occur  once,  even  if  a  player  returns  to  the  position  before 
the  castle;  more  importantly,  there  are  no  legal  moves  if  50  moves  have  been  made  without 
a  capture  or  pawn  advancement  (this  is  what  makes  chess  a  finite  game).  A  move  contains 


43 


enough  information  so  that  it  is  possible  to  compute  all  successor  boards,  that  is,  those 
resulting  from  legal  moves. 

The  game  tree  is  a  useful  concept  in  exploring  game-playing  strategies.  This  is  a 
finite  tree  (for  finite  games  like  chess)  whose  nodes  are  board  positions,  and  whose  branches 
are  all  possible  complete  games.  A  terminal  node  of  the  tree  ends  in  either  a  win  for  White 
or  Black,  or  a  draw.  The  game-tbeontic  value  of  a  node  for  a  player  is  either  1  (a  win), 
0  (a  draw),  or  -1  (a  loss),  based  on  whether  that  player  can  force  a  win  or  a  draw,  or 
his  opponent  can  force  a  win.  We  use  the  predicate  M{p,  b,  k,l,r)  to  mean  that  board  b 
has  value  k  for  player  p.  The  argument  /  is  a  depth-of-search  indicator,  and  shows  the 
maximum  depth  of  the  game  tree  that  the  value  is  based  on.  We  include  the  argument  r 
so  that  jU  can  represent  heuristic  information  about  the  value  of  a  node;  when  r  =  f,  i 
is  the  player’s  subjective  estimate  of  the  value  of  the  node,  /.e.,  he  has  not  searched  to  all 
terminal  nodes  of  the  game  tree.  If  r  =  t,  then  k  is  the  game- theoretic  value  of  the  board. 

We  take  the  formal  interpretation  of  boards,  players,  and  the  M  predicate  to  be 
the  game  of  chess,  so  that  Le  is  a  partially  interpreted  language.  The  rules  of  the  game 
of  chess  strictly  specify  what  the  game  tree  and  its  associated  values  will  be;  hence,  each 
predication  M{p,b,k,l,t)  or  its  negation  is  a  valid  consequence  of  these  interpretations. 
.\ny  agent  who  knows  the  rules  of  chess,  and  who  has  the  concept  of  game  trees,  will 
know  the  game- theoretic  value  of  every  node  if  his  beliefs  are  consequentially  closed.  In 
particular,  he  will  believe  either  A/(5u,,  /,  1,  k,  t)  or  1, 1,  k,  t),  where  I  is  the  initial 

board;  and  so  he  will  know  whether  White  has  an  initial  forced  win  or  not. 


We  represent  agents’  knowledge  of  chess  by  giving  tableau  rules  for  Le.  The  rules 
Tc  presented  below  are  one  possible  choice. 

Q.j^  .  _ r  =>  M[p,b,k,l,r),A _ 

‘  r  ^  A/(p,6i,ii,/i,ri),  A  r  =»  A/(p,62>^3if2)''2).-^  "■  T  =>  A/(p,irt, r„),  A’ 

where  bi-bn  are  all  the  legal  successor  boards  to  b 
p’s  opponent  is  to  move  on  6 
k  is  the  minimum  of  ki-kn 
I  is  1-i-  the  maximum  of  fi-/„ 
r  is  t  iff  all  of  rj-rn  are  t 
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Chx 


,  _ r  =>  M{p,b,k,l,r),A _ 

*■  '  r  ^  A/(p,6i,*l,/i,ri),  A  r  =>  M(p,b2,k2,l2,r2),  ^  T  ^  A/(p, 6„, fn).  A’ 

where  6i-6n  are  a]]  the  legal  successor  boards  to  b 
p  is  to  move  on  b 
k  is  the  maximum  of  ^i-^n 
I  is  1+  the  maximum  of  l\-ln 
r  is  t  iff  ail  of  r\-rn  are  t 

C/13  •  r  ^  A/(p,  6,i,  0,t),  A  ,  where  it  =  1  if  p  has  a  checkmate  on 

his  opponent  on  board  6;  J!r  =  0  if 
board  b  is  a  draw;  and  i  =  — 1  if  p’s 
opponent  has  a  checkmate. 

CA4  :  r  =>  A/(p,6,ir,  0,  f),  A  ,  where  k  is  any  number  between  —1 

and  1 

Ch\  axiomatizes  nodes  in  the  game  tree  where  p’s  opponent  moves.  The  value  of 
such  a  node  is  the  minimum  of  the  values  of  its  successor  nodes.  The  argument  /  is  the 
ma.ximum  depth  of  the  subtree  seartdied.  r  will  be  t  only  if  all  the  subtrees  have  been 
searched  to  leaf  nodes.  Ch2  is  similar  to  Ch\,  except  p  moves,  and  the  maximum  of  the 
successor  values  is  chosen. 


CA3  is  the  rule  for  terminal  nodes  of  the  tree.  C/14  is  a  rule  for  heuristic  evaluation 
of  any  node;  note  that  the  last  argument  to  M  is  f,  which  indicates  that  a  terminal  node 
has  not  been  reached.  Each  agent  may  have  his  own  particular  heuristics  for  evaluating 
nonterminal  nodes;  we  can  accommodate  this  by  changing  the  values  for  k  in  C/14. 


As  an  example  of  the  use  of  these  rules,  consider  the  following  tableau  proof. 

(5.1) 

=>  M{Stu,  1, 2,  t) 

^  61 , 1, 0, t)  =>  A/(iS'uji ^2>  1*  1>  1}  ^  A/(Sttj,  65, 1, 0,  t) 

X  ‘"'*2  =>  63,0,0,1}  ^  A/(5u„ 64,1,0,0  X 

X  X 


This  is  a  proof  that  the  board  6  has  a  value  1  for  White,  searching  to  all  terminal  nodes. 
Boards  61,  62,  and  63  all  have  value  1,  so  an  application  of  rule  CAj  yields  that  value  1  for  6 
(it  is  Black’s  turn  to  move  on  6).  Boards  61  and  65  are  terminal  nodes  that  are  checkmates 
for  White.  There  are  two  legal  moves  from  board  60;  one  ends  in  a  draw  (63),  the  other  in 
a  win  (64)  for  White.  Since  it  is  White’s  turn  to  move,  rule  CAo  applies. 
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The  structure  of  this  tableau  proof  mimics  exactly  the  structure  of  the  game  tree 
from  the  board  6,  Indeed,  for  any  subtree  of  the  complete  game  tree  of  chess  whose  root 
the  board  b  with  value  k  for  player  p,  there  is  a  corresponding  proof  of  M{p,  b,  k,  I,  t)  using 
the  rules  Tf  In  particular,  if  one  of  M(5u,, /,!,/,  t),  A/(5u,,/,0,/,  t),  or  A/(5u;, t) 
is  true,  there  is  a  proof  of  this  fact.  Hence  the  rules  Tc  are  sufficient  for  a  player  to 
reason  whether  White  has  a  forced  initial  win  or  not,  given  an  infinite  resource  bound  for 
derivations.  If  we  model  agents  as  having  the  rules  Tg,  so  that  Tc  C  t{u)  for  every  view 
!/,  the  conversation  presented  at  the  beginning  of  this  paper  would  make  sense:  each  agent 
would  believe  that  everyone  knew  whether  White  had  a  forced  initial  win. 

A  simple  modification  of  the  rules  Chi  ^^2  can  restrict  exploration  of  the  entire 
game  tree,  while  still  allowing  agents  to  reason  about  game  tree  values  using  the  heuristic 
axioms  Chi,  or  the  terminal  node  axioms  Chz  if  the  game  subtree  is  small.  All  that  is 
necessary  is  to  add  the  condition  that  no  rule  is  applicable  when  the  depth  I  is  greater  than 
some  constant  N.  Sw  would  still  be  able  to  reason  about  the  game  to  depths  less  than 
or  equal  to  N,  but  he  could  go  no  further.  In  this  way,  a  deductively  closed  system  can 
represent  a  resource-limited  derivation  process.  The  revised  rules  are 

Ch\  Chi,  with  the  condition  that  I  <  N. 

C/«2  Ch2i  with  the  condition  that  I  <  N. 

With  these  rules,  the  proof  of  (5.1)  would  still  go  through  for  W  >  2,  but  a  proof  of 
i\{(Sw,  I,  k,l,t)  could  not  be  found  if  N  were  low  enough  to  stop  search  at  a  reasonable 
level  of  the  game  tree. 

The  solution  to  the  chess  problem  illustrates  the  ability  of  the  deduction  model  to 
represent  resource  bounds  by  the  imposition  of  constraints  on  deduction  rules.  There  are 
other  workable  constraints  for  this  problem  besides  depth  cutoff:  for  example,  the  number 
of  nodes  in  the  tree  being  searched  could  be  kept  below  some  minimum.  Because  the 
structure  of  proofs  mimics  the  game  tree,  any  cutoff  condition  that  is  based  on  the  game 
tree  could  be  represented  by  appropriate  deduction  rules. 
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The  Not-So-Wise-Man  Problem 


For  this  problem  we  use  a  base  language  containing  only  the  three  primitive 
propositions  Pj,  P2,  and  P3.  P,-  expresses  the  proposition  that  wise  man  S{  has  a  white 
spot  on  his  forehead. 

In  the  initial  situation,  no  one  has  spoken  except  the  king,  who  has  declared  that  at 
least  one  spot  is  white.  Axioms  for  this  situation  are 


[Wl) 

P]  A  P2  A  P3 

(W2) 

[5o](P,  VP2VP3) 

(W.3) 

(P.  =  [5y]/’-)A[So](/^-=.[Sy]PJ, 

»  ^  3,  3  ^ 

(W4) 

(~>Pi  =>  [5y]-.p;)  A  [5o](-'F^-  3  (Syl-ip;-), 

«  #  3,3 

(Cl) 

{Si:W2-4,Pj,Pk)P{  =  [Si]Pi, 

‘  7^  3,  k 

VFl  describes  the  actual  placement  of  the  dots.  W2  is  the  result  of  the  king’s  utterance:  it 
is  a  common  belief  that  at  least  one  spot  is  white.  W3  and  IV 4  are  schemata  expressing  the 
wise  men’s  observational  abilities,  including  the  fact  that  everyone  is  aware  of  each  other's 
capabilities.  Cl  is  the  circumscriptive  ignorance  axiom:  the  only  beliefs  a  wise  man  has 
about  the  color  of  his  own  spot  are  the  three  axioms  W2-W^,  plus  his  observation  of  the 
other  two  wise  men’s  spots. 


As  an  exercise  of  the  formalism,  especially  the  circumscription  rules,  let  us  show 
that  all  agents  are  ignorant  of  the  color  of  their  own  spot  in  the  initial  situation. 


<5.2) 


I2 


CircKy 


C  _ Cl  =>  ^[5, if;- _ 

^  [Si]Pi  3  {Si :  W2-4,  Py,  Pk)Pi  =>  HSi]Pi 
{Sr.W2-4,P,;Pk)Pi=>^[Si]Pi  =>[5.-]P.-,^{5.-]F>- 

jf-  W2-4,  Pj,  Pk  =>i  Pi  '  [Si]Pi  =>  [5;]P.- 


We  have  omitted  some  irrelevant  sentences  from  the  left  side  of  sequents  in  this  tableau. 
To  show  that  it  closes,  we  must  be  able  to  prove  that  there  is  no  set  of  sound  deduction 
rules  that  will  enable  S,-  to  deduce  Pi  from  W2,  W3,  W4,  Pj,  and  Pjt.  We  can  prove  this 
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for  any  set  of  sound  tableau  rules  by  showing  that  W2-4i,Pj,Pk  Pi  is  not  provable  in 
the  normal  modal  logic  /\4  (see  Section  4.4).  It  is  possible  to  find  a  J?'4-model  in  which 
the  sequent  IV2-4,  Pj,  Pj^  P^  is  false,  using  the  methods  of  Sato  [38];  hence  this  sequent 
is  not  provable  in  any  logic  of  BK. 

After  the  first  wise  man  has  spoken,  it  becomes  a  common  belief  that  he  does  not 
know  his  own  spot  is  white.  The  appropriate  axioms  are 

(ir5)  a[5o]-[5,]Pi 

In  this  new  situation,  all  the  wise  men  are  again  ignorant  of  their  own  spot’s  color;  we 
could  prove  this  fact,  showing  that  |-  C'2  ^  -i[5,-]P;,  in  a  manner  similar  to  the  proof  in 
(5.2).  $2  relates  his  failure  to  the  others,  and  the  new  situation  has  the  additional  axiom 

(IV6)  -<[S2]P2  ^  [■?oH(‘?2]^2  • 

The  third  wise  man  at  this  point  does  have  sufficient  cause  to  claim  his  spot  is  white,  but 
only  if  the  second  wise  man  is  indeed  wise,  and  the  third  wise  man  believes  he  is.  To  see 
how  this  comes  about,  let  us  prove  it  In  the  saturated  form  of  BK.  We  will  take  the  wise 
men  to  be  powerful  reasoners,  and  set  t(i/)  =  Tq  +  AK^^  +  CireK\  +CireK2,  for  views 
i/.  The  sequent  we  wish  to  prove  is  W\-^  ^  [•?3l'P3> 

(5.3) 

,  _ W2-6,Pt,f^,  /^3[53]^=>[53]P3 _ 

^  ^  W2-6,  Pi,  P2,  P3,  [53]/^2=>[53lP3 

P2^P2  r  ^  W2-6,  Pi,  P2,  P3,  [53IP2,  Pi  [^alPi  =>  [S^ 

X  Vy2-6.  Pu  P2,  Pi,  [gslPj.  => 

X  W2-6,  Pi  V  Po  V  P3,  P2,  Pi  =>3  P3 

This  part  of  the  proof  is  mostly  bookkeeping.  We  have  used  some  shortcuts  in  the 
proof,  omitting  some  obvious  steps  and  dropping  sentences  from  either  side  of  the  sequent 
if  they  are  not  going  to  be  used. 


wi-e  =»  [53]  P3 
W2-6,  Pi,  P2,P3^  [galPs 
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We  now  must  show  that  S^’s  belief  subsystem  can  prove  P3  from  the  assumptions 
W'2-6  and  from  the  belief  that  the  other  two  wise  men’s  dots  are  white  (note  that  we  are 
now  using  53’s  sequent  =*>3). 


1^2-6,  Pi  V  Pi  V  P!3,  P2,  Pi  ^3  P3 

_ ^  1^2-6,  Pj,  P2,  Pi  =>[52]Fi  ^3  F3 _ 

H/2-6,  Pi,  P2,  [52] Fi  ^3  Ps 

Pi  =>3  Pi  r  *  ^2-6,  Pi,  P2,  [52]Fi,  -P3  =>  [-^sHPa  =>3  P3 

X  =^>3  P3,-,P3  W2-6,  Pi,  P.2,  [52]Pi,  [52]^P3  ^3  P3 

*  P3=>3P3  1^2-6,  Pi,  Py,  [52]Pi,  [S2hPi  ^3  Pj,  [■g2lP2 

X  1^2-6,  Pi  V  P2  V  P3,  Pi,  ^P3  =>32  P2 


Note  the  atom  P3  on  the  right-hand  side  of  the  top  sequent;  it  is  equivalent  to  ->Pz 
on  the  left-hand  side,  i.e.,  the  assumption  that  53’s  spot  is  black.  The  sequent  proof  here 
mimics  the  third  wise  man’s  reasoning,  Suppose  my  spot  were  black  ....  Through  the 
observation  axiom  WA,  which  is  a  common  belief,  this  assumption  means  that  Sz  believes 
that  S2  believes  "'P3.  At  this  point,  S3  begins  to  reason  about  S2’s  beliefs.  Since,  by  IV'G, 
the  second  wise  man  is  unaware  of  the  color  of  his  own  spot,  a  contradiction  will  be  derived 
if  P2  follows  in  S2’s  belief  subsystem. 


(5.5) 


Cl 


VK2-6,  Pi  V  fS  V  P3,  Pi,  -1P3  ^32  P2 
W2S,  Pi,  ~'Pz,  ~'P3  ^  [‘S'i]~‘'P3  =>32  P'l 


’P3  ^32  “'^3 

X 


Ni 


Cl 


iy2-6.  Pi,  ->P3,  [Si]->P3  =>32  P2 


1^2-6,  Pi,  -<Pz,  (5i]-^P3,  -■P2  3  [5i]--»P2  =»32  P2 


=*■32  P2>~‘P2 


P2 


VV2~6,  Pi,  -'P3,  {S1I-1P3,  [Si]-^P2  =>32  P2 


32  P2  xifCB  ^2-6,  Pi,  ->Pz,  {Si]-»P3,  (Si]-iP2  =>32  P2i  [■S'llPl^  [‘^ihPl 

1^2-6,  Pi  V  P2  V  P3,  -1P2,  -tPs  ^321  Pi 


S2’s  reasoning  (in  S3’3  view)  takes  the  assumption  that  the  third  wise  man’s  spot  is 
black  and  asks  what  the  effect  would  be  on  the  first  wise  mao  Si.  Since  Si  is  also  ignorant 
of  the  color  of  his  own  spot,  a  contradiction  will  ensue  if  the  first  wise  man  can  prove  that 
his  own  spot  is  white,  imder  the  assumption  -'Pz.  The  remainder  of  the  proof  is  conducted 
in  the  view  321. 
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(S.6) 


Z?2 


W2-Q^  Pi  V  P2  V  ~‘P2i  =>321 

_  ^  VV2-6,  Fi  V  P2  V  =>a2i  Pi .  P2,  fh _ 

f*!  ^*1.  Pz,  Pz  Pz  =»-32i  Pi,  Pz,  Pz  Pz  =>321  ^1.  Pz,  Pz 

XXX 


In  pursuing  this  proof,  we  have  assumed  that  the  second  wise  man  is  indeed  wise. 
There  are  several  places  in  which,  with  slightly  less  powerful  deduction  rules  for  the  view 
.32,  th6  proof  would  break  down.  Each  of  these  corresponds  to  one  of  the  two  types  of 
incompleteness  that  we  identified  in  the  statement  of  the  problem:  relevance  incompleteness 
and  fundamental  logical  incompleteness. 


Consider  first  the  notion  that  Sj  is  not  particularly  good  at  reasoning  about  what 
other  agents  do  not  believe,  a  case  of  fundamental  logical  incompleteness.  One  way  to 
capture  this  would  be  to  weaken  the  rule  N2  in  the  following  manner: 


r,-*?  =»32  A 
r  =>32  p,^  ’ 


where  p  contains  no  belief  operators 


The  modified  rule  would  not  allow  deductions  about  what  agents  do  not  know.  In 
particular,  it  would  not  allow  the  transfer  of  the  sentence  ~'{5i|Pi  to  the  left-hand  side  of 
the  sequent,  a  crucial  step  in  the  tableau  (5.5)  for  the  view  ^33. 


Note  that  the  modified  rule  still  allows  deductions  about  what  other  agents  do 
believe.  For  instance,  if  S2  were  asked  whether  Si's  believing  Pi  followed  from  his  believing 
-1P2  and  -'P3,  S2  would  say  “yes,”  even  with  the  logically  incomplete  rule  IVj  (as  in  tableau 
(5.6)  above). 


A  more  drastic  case  of  logical  incompleteness  would  result  if  53  simply  did  not  reason 
about  the  beliefs  of  other  agents  at  all.  In  that  case,  one  would  exclude  the  rule  from 

S2’s  deduction  structure.  Again,  the  proof  would  not  go  through,  because  the  attachment 
rule  could  not  be  applied  in  the  tableau  (5.5). 


The  notion  of  relevance  incompleteness  emerges  if  the  not-so-wise-man  S2  does  not 
consider  all  the  information  he  has  available  to  answer  the  king.  For  example,  he  may 
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think  that  the  observations  of  other  agents  are  not  relevant  to  the  determination  of  his 
own  spot,  since  the  results  of  those  observations  are  not  directly  available  to  him.  The 
observ'ational  axioms  W3  and  Wi  enter  into  the  proof  tableau  (5.5)  in  two  places.  Both 
times  the  rule  /o  is  used  to  break  statements  of  the  form  p  ^  [5]p  into  their  component 
atoms.  Preventing  the  decomposition  of  and  W4  effectively  prevents  S2  from  reasoning 
about  the  observations  of  other  agents.  A  weakened  version  of  I2  for  doing  this  is: 


r,P  q  ^32  ^ 

r  ^32  pi^  r,g  ^32 


where  p  and  q  are  both  modal  or 
both  nonmodal. 


This  rule  is  actually  weaker  than  required  for  the  purpose  we  have  in  mind.  Consider 
the  observation  axiom  -1P3  =  [5i]-'P3.  There  are  two  ways  S2  could  use  this  axiom.  If  So 
believes  ->^3,  he  could  conclude  that  Si  does  also.  This  is  not  the  type  of  deduction  we  wish 
to  prevent,  since  it  means  that  S2  attributes  beliefs  to  other  agents  based  on  his  own  beliefs 
about  the  world.  On  the  other  hand,  the  axiom  -'P2  =>  [5i]-'P2  is  'ised  in  a  conceptually 
different  fashion.  Here  it  is  the  contrapositive  implication:  if  5i  actually  does  not  believe 
■oPn,  then  Po  must  hold.  The  way  this  shows  up  in  the  proof  tableau  (5.5)  is  that  ->^3 
appears  as  an  initial  assumption  on  the  sequent  1^2-5,  Pi,  -1P3  ^32  Pz,  while  Po  is  a  goal 
to  be  proved. 


To  capture  the  notion  of  using  an  implicational  sentence  in  one  direction  only,  we 
would  have  to  complicate  the  deduction  rules  by  introducing  asymmetry  between  the  left 
and  right  sides  of  the  sequent.  Tliis  is  one  of  the  major  strategies  used  by  commonsense 
theorem  provers  of  the  PLANNER  tradition  (Hewitt  [12]  originated  this  theorem-proving 
method).  Rather  than  having  implicational  rules  of  the  form  I2,  typical  PLANNER-type 
systems  use  something  like  the  following  rule. 

PI,  r,p,p3g=>  A 

r,p,g,p  =>  q  ^  A 

The  implicational  sentence  is  used  in  one  direction  only  in  PI.  If  it  is  desired  to  make 
contrapositive  inferences,  then  the  contrapositive  form  of  the  implication  must  be  included 
explicitly.  The  construction  of  PLANNER-type  deduction  rules  within  the  tableau  frame¬ 
work  allows  a  much  finer  degree  of  control  over  the  inference  process.  A  full  exposition 
of  such  a  system  is  beyond  the  scope  of  this  paper,  the  interested  reader  is  referred  to 
Konolige  [21]. 
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In  sum,  we  have  shown  that  it  is  possible  for  the  deduction  model  to  represent  the 
situation  in  which  not-so-wise>man  has  less  than  perfect  reasoning  ability,  preventing  the 
third  wise  man  from  figuring  out  the  color  of  his  own  spot.  Both  relevance  incompleteness 
and  fundamental  logical  incompleteness  can  be  captured  by  using  appropriate  rules  for 
r(32). 
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6.  Other  Formal  Approaches  to  Belief 


How  does  the  deduction  model  and  its  logic  B  compare  to  other  formal  models  and 
logics  of  belief?  We  examine  two  alternative  approaches  in  this  section:  modal  logics  based 
on  a  Hintikka/Kripke  possible-worlds  semantics,  and  several  different  first-order  formaliza¬ 
tions  that  treat  beliefs  as  sentences  in  an  internal  language. 

6.1.  The  Possible- Worlds  Model 

The  possible- worlds  model  of  belief  was  initially  developed  by  Hintikka  in  terras  of 

sets  of  sentences  he  called  mode/  sets.  Subsequent  to  Kripke’s  introduction  of  possible 

worlds  as  a  uniform  semantics  for  various  modal  systems,  Hintikka  rephrased  his  work  in 

these  terms  (see  Hintikka  [14]).  The  basic  idea  behind  this  approach  is  that  the  beliefs  of 

an  agent  are  modeled  as  a  set  of  possible  worlds,  namely,  those  that  are  compatible  with 

his  beliefs.  For  e.xample,  an  agent  who  believes  the  sentences 

.  Some  of  the  artists  are  beekeepers. 

'  ’  '  All  of  tbe  beekeepers  are  chemists. 

would  have  his  beliefs  represented  as  the  set  of  possible  worlds  in  which  some  artists  are 
beekeepers' and  all  beekeepers  are  chemists. 

Representat/ona/  issues 

In  a  possible  world  for  which  the  sentences  (6.1)  are  true,  anything  that  is  a  valid 
consequence  of  (6.1)  must  also  be  true.  There  can  be  no  possible  world  in  which  some  artists 
are  beekeepers,  all  beekeepers  are  chemists,  and  no  artists  are  chemists;  such  a  world  is 
a  logical  impossibility.  If  beliefs  are  compatible  with  a  set  of  possible  worlds  (i.e.,  true  of 
each  such  possible  world),  then  every  valid  consequence  of  those  beliefs  is  also  compatible 
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with  the  set.  Thus  one  of  the  properties  of  the  possible-worlds  model  is  that  an  agent 
will  believe  all  consequences  of  his  beliefs  -  the  model  is  consequentially  closed.  Hintikka, 
recognizing  this  as  a  serious  shortcoming  of  the  model,  clzumed  only  that  it  represented  an 
idealized  condition;  an  agent  could  justifiably  believe  any  of  the  consequences  of  his  beliefs, 
although  in  any  given  situation  he  might  have  only  enough  cognitive  resources  to  derive  a 
subset  of  them. 


The  assumption  of  consequential  closure  limits  the  ability  of  the  possible-worlds 
model  to  represent  the  cognitive  state  of  agents.  Consider,  for  example,  the  problem  of 
representing  the  mental  state  of  agents  as  described  by  belief  reports  in  a  natural  language. 
Suppose  the  state  of  John’s  beliefs  is  at  least  partially  given  by  the  sentence 


(6.2) 


John  believes  that  given  the  ru/es  of  chess,  White  has  a  forced  initial 
win. 


Since  the  statement,  given  the  rules  of  chess,  White  has  a  forced  initial  win  is  either  a 
tautology  or  inconsistent,  this  would  be  equivalent  in  the  possible- world  model  to  one  of 
the  following  belief  reports: 


(6.3) 


a.  John  believes  t. 

b.  John  believes  everything. 


Clearly  this  is  wrong;  if  it  turns  out  that  John’s  belief  in  White’s  forced  initial  win  is 
correct,  John  has  a  good  deal  of  information  about  chess,  and  we  would  not  want  to  equate 
it  to  the  tautology  t.  On  the  other  hand,  if  John’s  belief  is  false  and  no  such  strategy  for 
White  exists,  it  is  not  necessarily  the  case  that  all  of  his  beliefs  about  other  aspects  of  the 
world  are  incoherent.  Yet  there  are  no  possible  worlds  compatible  with  a  false  belief,  and 
so  every  proposition  about  the  world  mtist  be  a  belief. 

The  representational  problems  of  the  possible-worlds  approach  stem  from  its  treat¬ 
ment  of  belief  as  a  relation  between  an  agent  and  a  proposition  (r.e.,  a  set  of  possible  worlds). 
.All  logically  equivalent  ways  of  stating  the  same  proposition,  no  matter  how  complicated, 
count  as  a  report  of  the  same  belief.  By  contrast,  the  deduction  model  treats  belief  as 
a  relation  between  an  agent  and  the  statement  of  a  proposition,  so  that  two  functionally 
different  beliefs  can  have  the  same  propositional  content. 


There  is  a  large  philosophical  literature  on  the  problems  of  representing  propositional 
attitudes  using  possible  worlds.  Perry  (in  [34j)  gives  an  account  of  some  of  the  more  subtle 


54 


problems  inherent  in  equating  belief  states  with  propositions;  his  analysis  does  not  depend 
on  consequential  closure.  Barwise  (in  [2])  critiques  consequential  closure  in  possible- worlds 
models  of  perception.  By  comparison,  a  good  account  of  the  relative  advantages  of  a 
symbol-processing  approach  to  representing  belief  can  be  found  in  Moore  and  Hendrix  (in 
(31]). 

The  Correspondence  Property 

It  is  reasonable  to  ask  how  the  deduction  and  possible-worlds  models  compare  in 
respects  other  than  the  assumption  of  consequential  closure.  That  is,  are  the  saturated 
deduction  models  Dt(L,p)  (whose  rules  are  consequentially  complete)  significantly  different 
from  possible-worlds  models  for  the  purpose  of  representing  belief? 

The  last  phrase,  “for  the  purpose  of  representing  belief,”  is  important.  The  two 
models  are  composed  of  different  entities  (expressions  vs.  propositions),  so  we  can  always 
use  a  language  that  distinguishes  these  entities,  and  has  statements  that  are  valid  in  one 
model  and  not  the  other.  So  the  answer  to  this  question  depends  on  the  type  of  language 
used  to  talk  about  the  models.  Fortunately,  the  language  standardly  used  to  axiomatize 
possible-worlds  models  is  the  same  as  that  of  B:  a  modal  calculus  containing  atoms  of  the 
form  [5jp,  in  which  p  refers  to  a  proposition.*  Thus  it  is  possible  to  compare  the  possible- 
worlds  and  deduction  models  by  comparing  their  axiomatizations  in  modal  logic.  We  have 
proven  the  following  general  property  about  the  two  approaches. 

Correspondence  Property.  For  every  modal  logic  of  belief  based  on  Kripke  possible- 
worlds  models,  there  exists  a  corresponding  deduction  model  logic  family  with  an 
equivalent  saturated  logic. 

The  correspondence  property  simply  says  that  possible- worlds  models  are  indistinguishable 
from  saturated  deduction  models  from  the  point  of  view  of  modal  logics  of  belief.  To  the 
author’s  knowledge,  this  is  the  first  time  that  the  symbol-processing  and  possible-worlds 
approaches  to  belief  have  been  shown  to  be  comparable,  in  that  the  possible-worlds  model 

*  Historically,  the  axiomatization  of  modal  systems  preceded  Kripke's  introduction  of  a  unifying  possible- 
worlds  semantics. 
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is  equivalent  to  the  limiting  case  of  a  symbol-processing  model  with  logically  complete 
deduction. 

Although  space  is  too  short  here  to  give  a  full  proof  of  this  claim,  we  will  give  an 
overview  of  the  most  important  of  the  propositional  modal  logics  with  a  possible-worlds 
semantics,  and  their  corresponding  deductive  belief  logics  (a  full  exposition  and  proofs  of 
results  mentioned  here  are  in  Konolige  [2l]). 


Modal  calculi  for  the  possible-worlds  model  differ,  depending  on  the  particulars  of 
their  intended  domains.  For  propositional  modal  calculi,  these  particulars  center  around 
whether  knowledge  or  belief  Is  being  axiomatized,  and  what  assumptions  are  made  about 
self-beliefs  or  self-knowledge  (a  survey  of  these  calculi  may  be  found  in  Hughes  and  Cresswell 
[l3]).  The  standard  propositional  modal  calculi  contain  a  single  modal  operator  (which  we 
write  here  as  [S])  and  are  expressed  as  Hilbert  systems.  Their  rules  of  inference  are  modus 
ponens  (from  p  and  p^q,  infer  q)  and  necessitation  (from  p,  infer  [5|p).  Axioms  are  taken 
from  the  following  schemata, 

p,  where  p  is  a  tautology 
M2.  [5l(p  =>  7)  =  ((5]p  =)  [S]q) 

A/3.  [5]p  3  p 
A/4.  [5]p  =  [5![Sjp 
A/5.  ~'[‘5]p  3  (S]-i(5jp 

A/1  are  the  purely  propositional  axioms.  A/2,  also  called  the  distribution  axioms,  allow 
modus  ponens  to  operate  under  the  scope  of  the  modal  operator.  A/3  are  a.xioms  for  knowl¬ 
edge:  all  knowledge  is  true.  A/4  and  A/5  are  called  the  positive  and  negative  introspection 
axioms,  respectively:  if  an  agent  believes  p,  then  he  believes  that  he  believes  it  (.iV/4);  if  he 
doesn’t  believe  p,  then  he  believes  that  he  doesn’t  believe  it  (A/5). 

Any  modal  calculus  that  uses  modus  ponens  and  necessitation,  and  includes  all 
tautologies  and  the  distribution  axioms,  is  called  a  normal  modal  calculus.  Normal  modal 
calculi  have  the  following  interesting  property  (see  Boolos  [3]):  if  p  =  7  is  a  theorem,  then 
so  is  [S]p  3  [5J7.  Interpreting  the  modal  operator  [5]  as  belief,  this  asserts  that  whenever 
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q  is  implied  by  p,  an  agent  S  who  believes  p  will  also  believe  q.  As  expected,  normal  modal 
calculi  assume  consequential  closure  when  the  modal  operator  is  interpreted  as  belief. 

The  simplest  normal  modal  calculus  is  K,  which  contains  just  the  schemata  Ml  and 
A/2.  To  a.xiomatize  knowledge,  M3  is  included  to  form  the  calculus  T.  Assumptions  about 
self-knowledge  lead  to  the  calculi  54  (T*  •¥  MA)  and  55  (54-1-  A/5).  McCarthy  (in  [24]  and 
[25])  was  the  first  to  recognize  the  utility  of  modal  calculi  for  reasoning  about  knowledge  in 
Al  systems,  and  defined  three  calculi  that  were  extensions  to  T,  54,  and  55,  allowing  belief 
operators  for  multiple  agents.  Sato  ([38])  has  a  detailed  analysis  of  these  calculi  as  Gentzen 
systems,  and  calls  them  KZ,  K4,  and  K5,  respectively.  He  also  gives  decision  procedures 
for  these  logics.  K4  is  the  calculus  used  by  Moore  in  his  dissertation  on  the  interaction  of 
knowledge  and  action  ([29]). 

The  so-called  weak  analogs  to  54  and  55  are  formed  by  omitting  the  knowledge 
axiom  A/3  (this  terminology  is  introduced  by  Stalnaker  [41]).  The  weak  versions  are  ap¬ 
propriate  for  axiomatizing  belief  rather  than  knowledge,  since  beliefs  can  be  false.  Levesque 
[22]  has  an  interesting  dissertation  b  which  he  explores  the  question  of  what  knowledge 
a  data  base  can  have  about  its  own  information.  Because  he  makes  the  assumption  that 
a  data  base  has  complete  and  accurate  knowledge  of  its  own  contents,  the  propositional 
calculus  he  arrives  at  is  weak  55,  with  the  addition  of  a  consistency  schema  [5]p  3  -i[5]-ip. 


How  does  the  family  of  logics  B  compare  with  these  propositional  modal  calculi? 
As  with  the  possible-worlds  logics,  the  deductive  belief  logics  formed  from  B  will  depend 
on  the  assumptions  that  are  made  about  self-beliefs.  In  this  paper  we  have  developed  the 
logic  family  BK,  which  assumes  that  an  agent  has  no  knowledge  of  his  own  beliefs.  The 
saturated  Fogic  BK«,  restricted  to  a  single  agent,  is  provably  equivalent  to  K,  the  weakest 
of  the  possible-worlds  belief  calculi. 

We  have  developed  a  theory  of  introspection  within  the  deduction  model  framework 
that  accounts  for  varying  degrees  of  self-knowledge  about  one’s  own  beliefs.  This  theory 
is  based  on  the  idea  that  an  agent’s  belief  subsystem  can  query  a  model  of  itself  (an 
introspective  belief  subsystem)  to  answer  question  of  self-belief.  Depending  on  constraints 
placed  on  the  introspective  belief  subsystem,  it  is  possible  to  arrive  at  any  one  of  eight 
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different  logic  families.  Two  of  these,  BS4  and  BS5,  have  saturated  logics  that  are  equivalent 
in  the  single-agent  case  to  the  modal  systems  weak  54  and  weak  55. 


While  we  have  been  interested  in  the  concept  of  belief  throughout  this  paper,  it  is 
possible  to  define  a  deductive  belief  logic  based  on  the  related  concept  of  knowledge.  One 
property  that  distinguishes  knowledge  from  belief  is  that  if  something  is  known  it  must  be 
true,  whereas  beliefs  can  be  false.  The  appropriate  tableau  axiom  for  knowledge  is 


E,r,[5v]r=>A 


Adding  Kq  to  B  forms  the  logic  family  K.  Particularizations  of  K  with  varying  degrees  of 
self-knowledge  correspond  to  the  propositional  modal  systems  T,  54  and  55. 


We  summarize  these  results  in  the  following  table. 


Belief 


Knowledge 


Normal  Modal 
Calculus 

Deduction  Model 
Family 

K 

BK 

weak  54 

BS4 

weak  55 

BS5 

T 

KT 

54 

KS4 

55 

KS5 

6.2.  Syntactic  Logics  for  Belief 

There  are  a  number  of  first-order  formalizations  of  belief  or  knowledge  in  the  symbol¬ 
processing  tradition  that  have  been  proposed  for  AI  systems.  We  have  labeled  these  “syn¬ 
tactic”  logics  because  their  common  characteristic  is  to  have  terms  whose  intended  meaning 
is  an  expression  of  some  object  language.  The  object  language  is  either  a  formal  language 
(e.g.,  another  first-order  language)  or  an  internal  mental  language.  The  logic  B  is  also  a 
syntactic  logic,  although  it  uses  a  modal  operator;  the  argument  of  the  operator  denotes  a 
sentence  in  the  internal  language.  We  have  chosen  to  use  a  modal  language  for  B  because 
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it  has  a  relatively  simple  syntax  compared  to  first-order  formalizations.  It  is  also  less  e.\- 
pressive,  in  that  quantification  over  sentences  of  the  object  language  is  not  allowed  by  the 
modal  syntax. 

McCarthy  [26]  has  presented  some  incomplete  work  in  which  individual  concepts  are 
reified  in  a  first-order  logic.  Exactly  what  these  concepts  are  is  left  deliberately  unclear, 
but  in  one  interpretation  they  can  be  taken  for  the  internal  mental  language  of  a  symbol¬ 
processing  cognitive  framework.  He  shows  how  the  use  of  such  concepts  can  solve  the 
standard  representational  problems  of  knowledge  and  belief,  e.g.,  distinguishing  between 
de  dicto  and  de  re  references  in  belief  sentences. 

system  that  takes  seriously  the  idea  that  agent’s  beliefs  can  be  modeled  as  the 
theory  of  some  first-order  language  is  proposed  by  Konolige  [19],  A  first-order  metalanguage 
is  used  to  axiomatize  the  provability  relation  of  the  object  language.  To  account  for  nested 
beliefs,  the  agent’s  object  language  is  itself  viewed  as  a  metalanguage  for  another  object 
language,  and  so  on,  thereby  creating  a  hierarchy  of  metalanguage/object  language  pairs. 
Perils  [33]  presents  a  more  psychologically  oriented  first-order  theory  that  contains  a.\ioms 
about  long-  and  short-term  memory.  The  ontology  is  that  of  an  internal  mental  language. 


These  a.\iomatic  approaches  are  marred  by  one  or  both  of  two  defects  -  the  lack 
of  a  coherent  formal  model  of  belief,  and  computational  inefficiency.  Regarding  the  first 
one:  the  vagueness  of  the  intended  model  often  makes  it  difficult  to  claim  that  the  given 
axioms  are  the  correct  ones,  since  there  is  no  formal  mathematical  model  that  is  being 
axiomatized.  In  arriving  at  the  deduction  model  of  belief,  we  have  tried  to  be  very  clear 
about  what  assumptions  were  being  made  in  abstracting  the  model,  how  the  model  could 
fail  to  portray  belief  subsystems  accurately,  and  so  on.  In  contrast,  the  restrictions  these 
syntactic  systems  place  on  belief  subsystems  are  often  obscure.  What  type  of  reasoning 
processes  operate  to  produce  consequences  of  beliefs?  How  are  these  processes  invoked? 
What  is  the  interaction  of  the  belief  subsystem  with  other  parts  of  the  cognitive  model? 
These  types  of  questions  are  begged  when  one  simply  writes  first-order  axioms  and  then 
tries  to  convey  an  intuitive  idea  of  their  intended  content.  (To  some  extent  this  criticism  is 
not  applicable  to  the  formalism  of  Konolige  in  [19],  because  here  the  intended  belief  model 
is  explicitly  stated  to  be  a  first-order  theory). 
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A  second  shortcoming  is  that  efficient  means  of  deduction  for  the  syntactic  axioma- 
tizations  are  not  provided.  As  we  have  mentioned,  a  system  that  is  actually  going  to  rea¬ 
son  about  belief  by  manipulating  some  formalization  can  encounter  severe  computational 
problems.  Many  of  the  assumptions  incorporated  into  the  deduction  model,  especially 
the  closure  property,  were  made  with  an  eye  towards  deductive  efficiency.  The  end  result 
is  a  simple  rule  of  inference,  the  attachment  rule  A,  that  has  computationally  attractive 
realizations.*^  On  the  other  hand,  formalizations  that  try  to  account  for  complex  proce¬ 
dural  interactions  (as  in  Perlis’s  theory  of  long-  and  short-term  memory),  or  that  use  a 
metalanguage  to  simulate  a  proof  procedure  at  the  object  language  level  (as  in  Konolige 
[19]),  have  no  obvious  computationally  efficient  implementation. 

’  Several  efficient  proof  methods  are  given  in  Konolige  [21]:  a  decision  procedure  for  propositional  BK 
based  on  the  Davis-Putnam  procedure  (see  Chang  and  Lee  [Sj),  which  is  sufficient  to  solve  the  Wise  Man 
Puzzle  automatically;  a  resolution  method  for  the  quantifying-in  form  of  B;  and  a  PLANNER-type  deduction 
system. 
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7. 


Conclusion 


We  have  explored  a  formalization  of  the  symbol-processing  paradigm  of  belief  that 
we  call  the  deduction  model.  It  is  interesting  that  the  methodology  employed  was  to  ex¬ 
amine  the  cognitive  structure  of  AI  planning  systems.  This  methodology,  which  we  might 
term  e.Yperimen  tai  robot  psycboJo^,  offers  some  distinct  advantages  over  its  human  coun¬ 
terpart.  Because  the  abstract  design  of  such  systems  is  open  and  available,  it  is  possible  to 
identify  major  cognitive  structures,  such  as  the  belief  subsystem,  that  influence  behavior. 
Moreover,  these  structures  are  likely  to  be  of  the  simplest  sort  necessary  to  accomplish 
some  task,  without  the  synergistic  complexity  so  frequently  encountered  in  studies  of  hu¬ 
man  intelligence.  The  design  of  a  robot’s  belief  subsystem  is  based  on  the  minimum  of 
assumptions  necessary  to  ensure  its  ability  to  reason  about  its  environment  in  a  productive 
manner,  namely,  it  incorporates  a  set  of  logical  sentences  about  the  world,  cind  a  theorem* 
proving  process  for  deriving  consequences.  The  deduction  model  is  derived  directly  from 
these  assumptions. 

The  deduction  model  falls  within  that  finely  boimded  region  between  formally  tract¬ 
able  but  oversimplified  models  and  more  realistic  but  less  easily  axiomatized  views.  On 
the  one  hand,  it  is  a  generalization  of  the  formal  possible-worlds  model  that  does  not  make 
the  cissumption  of  consequential  closure,  and  so  embodies  the  notion  that  reasoning  about 
one’s  beliefs  is  resource-limited.  On  the  other  hand,  it  possesses  a  concise  axiomatization 
in  which  an  agent’s  belief  deduction  process  is  incorporated  in  a  direct  manner,  rather  than 
simulated  indirectly.  Thus,  the  deduction  model  and  its  associated  logic  B  lend  themselves 
to  implementation  in  mechanical  theorem-proving  processes  as  a  means  of  giving  .AI  systems 
the  capability  of  reasoning  about  beliefs. 
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